Scale Your Service on What Matters Autoscaling on Latency

1. Scale Your Service on What Matters: Autoscaling on Latency
2. Hello KubeCon! Thomas Rampelberg Software Engineer @ Buoyant Twitter: @grampelberg
3. Get your votes in http://kc.l5d.io
4. Autoscaling in Kubernetes - Cluster Autoscaler - Horizontal Pod Autoscaler - Vertical Pod Autoscaler
5. Horizontal Pod Autoscaler - metrics.k8s.io - custom.metrics.k8s.io - external.metrics.k8s.io
6. CPU is an approximation - Not every workload is CPU bound - Isn’t 100% utilization good? - CPUs are different in the cloud - Orchestrated environments are complex CPU utilization is wrong: http://bit.ly/cpu-wrong Utilization is useless as a metric: http://bit.ly/useless-metric
7. Memory is workload specific
8. What can you scale on?
9. Golden Signals - Latency - Traffic - Errors - Saturation http://bit.ly/golden-signals
10. Every request matters - Tail latency is important - Users see responses - Latency is not normally distributed http://bit.ly/latency-wrong
11. What is required? ☐ Measure the latency of a service ☐ Expose custom metrics ☐ Autoscale!
12. An open source service mesh and CNCF member project. ➢ ➢ ➢ ➢ ➢ ➢ 24+ months in production 2,500+ Slack members 7,500+ GitHub stars 40m+ DockerHub pulls 100+ contributors 400b+ requests/mo
13. What is Linkerd?
14. Architecture
15. Measure the latency of a service
16. What is required? ☐ Measure the latency of a service ☐ Expose custom metrics ☐ Autoscale!
17. What are custom metrics? http://bit.ly/k8s-aggregation
18. Prometheus Adapter histogram_quantile(0.99, sum(irate(<<.Series>>{<<.LabelMatchers>>, direction="inbound"}[5m])) by (le, <<.GroupBy>>)) http://bit.ly/k8s-adapter
19. Architecture
20. Expose custom metrics
21. What is required? ☐ Measure the latency of a service ☐ Expose custom metrics ☐ Autoscale!
22. Architecture
23. Autoscale!
24. What is required? ☐ Measure the latency of a service ☐ Expose custom metrics ☐ Autoscale!
25. Route Based Scaling - / - /vote - /vote/{editor}/minus - /vote/{editor}/plus - /health
26. Predictive Scaling predict_linear(v range-vector, t scalar)
27. Slides http://bit.ly/l5d-autoscale Code http://bit.ly/kubecon-auto Get Started! https://bit.ly/linkerd-get-started Prometheus Adapter http://bit.ly/k8s-adapter