eBay 孟凡杰&辛肖刚 基于kubernetes集群联邦和Istio的流量管理

CodeWarrior

2019/07/08 发布于 编程 分类

GIAC2019 

文字内容
1. Istio Kubernetes Federation 2019 skubernetes Federation Istio
3. 1
4. 3 n k 100K+ 4K+ k 15 Search Grid Hadoop 200K+ PoP Database Kubernetes 4.5PB Frontend VM
5. Smart DNS 1 2 Cross Region LB Cross Region LB Pod Cross Region LB Regional LB Regional LB Pod 3 Pod Pod Pod Regional LB Pod Pod Pod Pod
6. 2
7. Kubernetes k • d j k j j • k • u (lvs) Dev k (slc) k 01 (lvsaz01) r K8S cluste 01 (slcaz01) r K8S cluste 01 (lvsaz01) r K8S cluste 01 (slcaz01) r K8S cluste 02 (lvsaz02) r K8S cluste 02 (slcaz02) r K8S cluste 02 (lvsaz02) r K8S cluste 02 (slcaz02) r K8S cluste Prod Secure
8. Kubernetes • MVDGSPG GT • • j ( j j R F • • 3N DCN 2GFGSC K P • -B 2GFGSC K P • VDGSPG GT 3N DCN 2GFGSC K P s-B • • MVDGSPG GT • rG.C k HGFGSC GF o T PE E P S NNGS a
9. k j oSync Controller k ob Impersonate
10. AZ • j GIK P • j GIK P • f u HCKNVSG F OCKP f • j j a s
11. s • • • eBay MVDGSPG GT o AZk j om mRemote cluster Istio Gateway
12. Availability Zone Gate way Clu ster L4 N IPV ode S/B ird r Cluste e t o Rem L4 N ode IPVS/Bird L4 N o d IPVS/B e ird L4 Node IPVS/Bird L7 P o Env d oy L7 P od Envo y Hardware LB Hardware LB Hardware LB
13. Istio • 3C GXC ENVT GS • o -B KS VCN=GSWKEG Gateway Cluster TGSWKEG GPFR KP N ECN SGO G Remote Cluster Kube APIServer Kube APIServer Istio Control Plane Service: foo.ns1.svc.cluster1 Service: foo.ns1.svc.cluster2 pod1 pod2 route: - destination: host: foo.ns1.svc.cluster1 port: number: 80 weight: 50 - destination: host: foo.ns1.svc.cluster2 port: number: 80 weight: 50
14. 3
15. o - . =7- , ++ ++ • m - . , -WCKNCDKNK e .VTKPGTT a • • • MVDGSPG GT • 9COGTRCEG =GESG w = PE E P S NNGS • m G.C k r a a s o o2GFGSC GF DLGE
16. ApplicationInstance Application Federated Deployment Deployment Application Instance Application Sync To Federated Deployment Gateway Sync To Application Instance Federated AccessPoint VirtualService NameServive Federated AccessPoint DestinationRule Service Federation Member Cluster
17. Application y • Application • jwebserver m Application Application jgithub repo (github repo) y • Application y (owner, escalation) • web a (type, data classification) a
18. ApplicationInstance • ApplicationInstance r o j j Z Dev Staging Prod mApplicationInstance • • • s o Firewall s • Node Declare Dependencies • • Create Security Policy Developer LoadBalancer DNS Security Reviewer
19. FederatedDeployment - . =7- , ++ ++ • m e - . , -WCKNCDKNK .VTKPGTT a • • m • G.C k CRCEK r o j • a oa • a a SGRNKECT R F s o2GFGSC GF0GRN R F OGP SGRNKECT s
20. FederatedDeployment Scope: cluster/21 Replicas: 25 Scope: cluster/22 Replicas: 25 Scope: az/lvs02 Replicas: 50 A C Scope: cluster/24 Replicas: 25 Scope: az/rnoaz01 Replicas: 50 Scope: cluster/25 Replicas: 25 ScopedDeployment Template Scope: global/ebay Replicas: 150 Scope: az/slc07 Replicas: 50 B M Scope: cluster/23 Replicas: 50 GlobalDeploymentTemplate
21. FederatedAccessPoint s • FederatedAccessPoint • 2GFGSC GF-EEGTT KP j 0GT KPC K P VNG MVDGSPG GT TGSWKEG • eBayf sGateway KS VCN=GSWKEG 7) SVNG o a AZ a KP 5T K a 5 • o jb 2GFGSC GF-EEGTT • a VDGSPG GT =GSWKEG =RGE 5T K 3C GXC m • FederatedAccessPoint o rmHTTPS a VIP m j2GFGSC GF-EEGTT KP j
22. FederatedAccessPoint Scope: cluster/21 Scope: az/lvs02 Scope: cluster/22 A C Scope: cluster/24 Scope: az/rnoaz01 Scope: cluster/25 Scoped AccessPointSpec Scope: global/ebay Scope: az/slc07 B M Scope: cluster/23 Global AccessPointSpec(Service/Gateway/VirtualService/DestinationRule)
23. FederatedAccessPoint Spec gateway: apiVersion: networking.istio.io/v1alpha3 kind: Gateway metadata: name: vi-gateway spec: selector: istio: ingressgateway servers: - hosts: - vi.vip.ebay.com port: name: https-vi number: 443 protocol: HTTPS tls: mode: SIMPLE credentialName: "vi-certs" virtualService: apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: vi spec: gateways: - vi-gateway hosts: - vi.vip.ebay.com http: - match: - port: 443 uri: prefix: "/" route: - destination: host: visvc.vi.svc.25.tess.io port: number: 8082 weight: 50 - destination: host: visvc.vi.svc.26.tess.io port: number: 8082 weight: 49
24. NameService • NameService • Provider ZA/CName/PTR) • • oTTL • Smart DNS DNS
25. NameService Target: foo.ns1.svc.21.tess.io Weight: 25 Target: foo.ns1.svc.22.tess.io Weight: 25 FQDN NameRoute Alias Target: foo.ns1.svc.23.tess.io Weight: 25 Target: foo.ns1.svc.24.tess.io Weight: 25 Target: foo.ns1.svc.25.tess.io Weight: 25
26. 4
27. • 5 CNN EC K P 8CPIGS m 5 • = 5 m 1. 2 A0 s5 5 • VPPGN 5 5 • 0= • =9• 7 5 sv • 7 • h 5 j7) s o VG S V G E P S NNGS s.3 5 Z.KSF F
28. 7) • b • G.C • m a 7 5 N K CFGN 9 FG -IGP VPPGN 7) FZ5T K 3C GXC RDCEM KP GSHCEG • sI CE S 1PW jMVDGSPG GT F 3C GXC =GSWKEG 5 5 Fh j3 TGSWGS
29. ECMP vip1 router k8s API Server vip1 adv. L4 Cluster vip1 adv. vip2 adv. DSR K8S Node K8S Node K8S Pod K8S Pod Elb-Director Elb-Director netlink netlink Linux Kernel / ipvs Linux Kernel / ipvs L7 Cluster vip2 adv. client K8S Node K8S Node K8S Pod K8S Pod L7 Proxy L7 Proxy control path data path / traffic BGP
30. 5
31. • • VDGSPG GT p mg R • • • • 0= GTR PTG 7) F 7 • mt • • 7 • 7) s i MVDGSPG GT R F a CV TECNGS f
32. • m 8GT c • • 5 =GSWKEG 8GT s f T H XCSG N CFDCNCPEGS T NV K P = 1PW i • 5T K 7P % % + MVDGSPG GT 0GW = CIKPI SG S F • % % • 5T K p l b OGT f SC G NKOK p SCEKPI TGSWKEG ISCR G.C 5T K
33. 1 - 1 45 593 2019.6.23