TalkingData 宋净超 - 从Kubernetes到Cloud Native——云原生应用之路_部分2

龚平萱

2017/12/18 发布于 技术 分类

ArchSummit全球架构师峰会是InfoQ中国团队推出的面向高端技术管理者、架构师的技术大会,参会者中超过50%拥有8年以上的工作经验。 ArchSummit秉承“实践第一、案例为主”的原则,展示新技术在行业应用中的最新实践,技术在企业转型中的加速作用,帮助企业技术管理者、CTO、架构师做好技术选型、技术团队组建与管理,并确立技术对于产品和业务的关键作用。

文字内容
1. 3UouM NJtRvN b 容器生态 b 4Nv?Ys b 持续t付 b 微服务
2. 3UouM NJtRvN 6NJtuZNs Container packaged Running applications and processes in software containers as an isolated unit of application deployment, and as a mechanism to achieve high levels of resource isolation. Dynamically managed Actively scheduled and actively managed by a central orchestrating process. Microservices oriented Loosely coupled with dependencies explicitly described (e.g., through service endpoints).
3. 3UouM NJtRvN QostNM YZoSNLts souZLN QttYs///LWLO.Ro
5. &2 OJLtoZs &. 基准y码 2. 依赖管理 (. 配置 ). 后端服务 *. 构建f发布f运行 +. 无状态进程 7. 端口绑定 -. 并发 .. 易处理 &0.开发环境l线i环境等价 &&.日志作o事件流 &2.管理进程 1MMRtRoWJU b 1P9声明管理 b 认证和授权 b 监控l告警 souZLN QttYs///&2OJLtoZ.WNt/
6. 8ow to LZNJtN J 3UouM NJtRvN JYY
7. MovN to 3UouM Automodel Resource Allocation(Before) sdl 4.Send tasks 1.Start master 2.Start worker 3.Connect master master Tasks queue 5.Assign tasks worker worker Solid workers as resource pool worker https://jimmysong.io'>https://jimmysong.io Automodel Resource Allocation(After) sdl 2.Send tasks 1.Start master 3.Keep watching pool master Tasks queue 6.Destroy worker/Release resource 5.Callback 4.Start worker worker CPUMemory Resource pool https://jimmysong.io'>https://jimmysong.io
8. 39/34 b :NWTRWs b 8JZKoZ b PowNZ4NC b 56K Edge node with a VIP 172.168.0.1 Running as DaemonSets Pull image Pod Pod Kubernetes cluster Kubernetes ɧUpdate DNS servicename   Template filebeat-test.yaml ɨCall ɣPush image ɤWith scripts ɥCreating serivce yaml ɦUpdate ingress ɢTrigger ingress.yaml ɡPush ɠCommit 8VHUGHƉQHG Service Name Resource Request Instance number GitURL Brunch Service type © Jimmy Song https://github.com/rootsongjc/kubernetes-handbook
9. 5MPN WoMN b 边缘节点 b 流量出入口 b Qost ! YJtQ Ingress 0Update 1Add a item ... ... Add a service VIP 172.20.0.119 Kubernetes Master/Node 172.20.0.113 Edge nodes Node 172.20.0.114 Node 172.20.0.115 Node 172.20.0.116 Kubernetes cluster © Jimmy Song https://github.com/rootsongjc/kubernetes-handbook
10. MoWRtoZRWP
11. ?utURWN • Docker in TalkingData • Containers • Why Kubernetes? • Microservices • Cloud Native • Service Mesh • Use Cases • Open Source
12. CNZvRLN MNsQ MRLZosNZvRLNs 7ovNZWVNWt oW KuKNZWNtNs bm用的基础设施层 b轻量级高性能网络y理 9stRo LRWTNZM b提供安全的、快速的、可靠地服务间通讯 b扩展TuKNZWNtNs的应用负载均衡机制f实现灰度发布 bDZJOORL MJWJPNVNWtg1P9网关 b?KsNZvJKRURtyg服务调用和性能分析 bPoURLy 5WOoZLVNWtg控制服务访问策略 b完全解耦r应用f应用可以无感知f加速应用的微服务和s 原生转型 bCNZvRLN 9MNWtRty JWM CNLuZRtyg安全保护 QttYs///PRtQuK.LoV/ZuWLoWMuRt/LoWMuRt
13. CNZvRLN MNsQ 9stRo FC LRWTNZM CNZvRLN MNsQn文网gQttY///www.sNZvRLNVNsQ.LW/
14. 9stRo vs LRWTNZM
15. 9stRo vs LRWTNZM Feature Istio Linkerd Deployment Envoy/ Sidecar DaemonSets Easy to use complicated easy Platform kuberentes kubernetes/ mesos/Istio/local Version 0.3.0 1.3.3 Production No Yes Istio Kubelet node Istio mixer Istio ingress App Envoy Kubelet node Istio egress Istio manager App Envoy Linkerd Kubelet node OG $SS Kubelet node OG $SS
16. 7ZJOJWJ
17. IRYTRW
18. GNJvN sLoYN
19. EsN 3JsNs • Docker in TalkingData • Containers • Why Kubernetes? • Microservices • Cloud Native • Service Mesh • Use Cases • Open Source
20. 7Rt?Ys b 9WOZustZJLtuZN Js LoMN b H1ML! H1ML! H1ML! b 7Rt vNZsRoW LoWtZoU source https://www.weave.works
21. 7Rt?Ys b i线/更新/j线 b H1ML! H1ML! H1ML! b 8NUV LQJZt
22. CYJZT oW KuKNZWNtNs Spark on Kubernetes with different schedulers Yarn Standalone Native Kubelet node Yarn Node NodeManager App1 Executor App2 Executor Kubelet node Spark worker pod App1 Executor App1 Executor App2 Executor Kubelet node App1 Executor Pod App1 Executor Pod App2 Executor Pod https://jimmysong.io
23. CYJZT oW KuKNZWNtNs wRtQ yJZW sLQNMuUNZ
24. CYJZT oW KuKNZNWtNs wRtQ WJtRvN sLQNMuUNZ b KuKNZWNtNs原生调度glyJZW、VNsos同级 b 资源隔离f粒度更细g以WJVNsYJLN来划分用户 监控的变革g单次任务资源计量 b ./spark-submit \ --deploy-mode cluster \ --class com.talkingdata.alluxio.hadooptest \ b 日志的变革gYoM的日志收集 --master k8s://https://172.20.0.113:6443 \ --kubernetes-namespace spark-cluster \ --conf spark.kubernetes.driverEnv.SPARK_USER=hadoop \ --conf spark.kubernetes.driverEnv.HADOOP_USER_NAME=hadoop \ --conf spark.executorEnv.HADOOP_USER_NAME=hadoop \ --conf spark.executorEnv.SPARK_USER=hadoop \ --conf spark.kubernetes.authenticate.driver.serviceAccountName=spark \ --conf spark.driver.memory=100G \ --conf spark.executor.memory=10G \ --conf spark.driver.cores=30 \ --conf spark.executor.cores=2 \ --conf spark.driver.maxResultSize=10240m \ --conf spark.kubernetes.driver.limit.cores=32 \ --conf spark.kubernetes.executor.limit.cores=3 \ --conf spark.kubernetes.executor.memoryOverhead=2g \ --conf spark.executor.instances=5 \ --conf spark.app.name=spark-pi \ --conf spark.kubernetes.driver.docker.image=spark-driver:v2.1.0-kubernetes-0.3.1-1 \ --conf spark.kubernetes.executor.docker.image=spark-executor:v2.1.0-kubernetes-0.3.1-1 \ --conf spark.kubernetes.initcontainer.docker.image=spark-init:v2.1.0-kubernetes-0.3.1-1 \ --conf spark.kubernetes.resourceStagingServer.uri=http://172.20.0.114:31000 \ ~/Downloads/tendcloud_2.10-1.0.jar
25. CYJZT oW HJZW F.C CYJZT oW KuKNZNWtNs Feature queue instance network heterogeneous security Yarn queue ExcutorContainer host no RBAC QttYs///PRtQuK.LoV/JYJLQN-sYJZT-oW-T-s/sYJZT Kubernetes namespace Executor Pod plugin yes ACL
26. ?utURWN • Docker in TalkingData • Containers • Why Kubernetes? • Microservices • Cloud Native • Service Mesh • Use Cases • Open Source