1. चԭ CSI ጱ Kubernetes ਂ‫ؙ‬ൊ կ୏‫ݎ‬ਫ᪢ ሴོ ᶆԯQingCloud
2. • ᙧว • ୏‫ ݎ‬CSI ൊկ • ᮱ᗟ CSI ൊկ • ᨶᰁᓕቘ • ઀๕
3. ᙧว Kubernetes ਂ‫ؙ‬ൊկ ਂ‫ؙ‬๐‫ ۓ‬GCE PD AWS EBS ... ሿํጱ Kubernetes ਂ‫ؙ‬ൊկ૪ᕪ ෫ဩჿ᪃ አಁ෭ፅीᳩጱᵱ࿢ ௒ᵱਖ਼ ਂ‫ؙ‬๐‫ ۓ‬Ө Kubernetes ੒ള̶
4. Kubernetes ਂ‫ؙ‬ൊկ‫ړ‬ᔄ Kubernetes In-tree ਂ‫ؙ‬ൊկ AWS GCE Ceph … Kubernetes Out-of-tree ਂ‫ؙ‬ൊկ AWS GCE Ceph … AWS GCE Ceph … AWS GCE Ceph …
5. In-tree vs Out-of-tree ‫ۑ‬ᚆ௔ ඪ೮ጱਂ‫ؙ‬ᔄࣳ In-tree ਂ‫ؙ‬ൊկ ਂ‫ڠܫؙ‬ୌ҅‫ڢ‬ᴻ҅೯᫹ ํᴴ҅ᴴ‫ګ‬አಁጱᭌೠ Out-of-tree ਂ‫ؙ‬ൊկ ਂ‫ڠܫؙ‬ୌ̵‫ڢ‬ᴻ޾೯᫹҅ ளᆙ‫ڠ‬ୌ޾‫ڢ‬ᴻᒵ‫ۑ‬ᚆ ग़໏҅୏‫ݎ‬ൊկ‫ݢܨ‬੒ള ฃᖌಷ௔ ൊկդᎱࣁ K8S դᎱՙପٖ҅ ӧฃᖌಷ ❖ ᕮᦞғ୏‫ֵ޾ݎ‬አ Out-of-tree ਂ‫ؙ‬ൊկ ൊկդᎱᇿᒈԭ Kubernetes҅ ‫ݢ‬ᇿᒈ຅ୌ̵‫҅૲ݎ‬ฃԭᖌಷ
6. Flex-volume vs CSI ❖ Out-of-tree ਂ‫ؙ‬ൊկ‫ړ‬ԅ Flex-volume ޾ CSI ӷᔄ ❖ Flex-volume: Kubernetes 1.2 ୏তඪ೮ ❖ ᮱ᗟ॔๥҅‫ۑ‬ᚆํᴴ ❖ CSI (Container Storage Interface): Kubernetes 1.9 ୏তඪ೮ ❖ ਻࢏ଘ‫ݣ‬ᶾऒૡӱຽ‫ٵ‬ ❖ ᮱ᗟᓌ‫҅׎‬ඪ೮਻࢏۸᮱ᗟ ❖ ‫ۑ‬ᚆ୩य़҅ඪ೮ਂ‫ܫؙ‬ᓕቘ҅ளᆙᓕቘᒵ‫ۑ‬ᚆ҅ࣁ೮ᖅள᭛‫઀ݎ‬Ӿ ❖ ᕮᦞғ୏‫ֵ޾ݎ‬አ CSI ਂ‫ؙ‬ൊկ
7. ፓຽғ୏‫ ݎ‬CSI ൊկ҅ଚ᭗ᬦ CSI ൊկਖ਼ਂ‫ؙ‬ ๐‫ۓ‬Ө Kubernetes ੒ള
8. • ᙧว • ୏‫ ݎ‬CSI ൊկ • ᮱ᗟ CSI ൊկ • ᨶᰁᓕቘ • ઀๕
9. CSI ൊկच๜ܻቘ ❖ CSI ൊկ᩸‫ک‬ಥӤ‫ސ‬ӥ֢አ҅Ӥ੶੒ള਻࢏ଘ‫҅ݣ‬ӥ੶඙֢ਂ‫ؙ‬๐‫ۓ‬ᒒ ❖ “ಥӤ”ғਫሿ CSI ള‫— ݗ‬-> ੒ള਻࢏ଘ‫҅ݣ‬ԅ਻࢏ଘ‫ݣ‬ᩙᚆ ❖ “‫ސ‬ӥ”ғ᧣አਂ‫ؙ‬๐‫ۓ‬ᒒ API —-> ඙֢ਂ‫ؙ‬ᩒრ CSI ൊկ ਻࢏ଘ‫ݣ‬ gRPC CSI ള‫ݗ‬ ਂ‫ؙ‬๐‫ۓ‬ᒒ API ਂ‫ؙ‬ᩒრ
10. ୏‫ ݎ‬CSI ൊկ CSI ਥොᥢ᝜ https://github.com/container-storage-interface/spec/blob/master/spec.md ❖ ਫሿ CSI ള‫ݗ‬ጱ‫ۑ‬ᚆ ❖ ਫሿ CSI ള‫ݗ‬ጱ଍ᒵ௔ ❖ ᒧ‫ ݳ‬CSI ᬬࢧ‫؀‬ᥢ᝜
11. CSI ള‫༷ݗ‬ᬿ ❖ ‫ړ‬ԅ Identity҅Controller҅Node ӣᔄള‫ݗ‬ ❖ ‫ړ‬ԅ Controller҅Node ӷᔄ๐‫ۓ‬ ‫ۑ‬ᚆ ള‫ݗ‬ Identity ൈᬿൊկच๜‫௳מ‬ ༄ၥൊկ؋଼ᇫா GetPluginInfo, Probe, GetPluginCapabilities Controller ‫ؙਂݻ‬๐‫ۓ‬ᒒ‫ݎ‬ᭆ೰ե ੒ਂ‫ܫؙ‬ᬰᤈᓕቘ CreateVolume, DeleteVolume, ControllerPublishVolume, ControllerUnpublishVolume, ControllerGetCapabilities … Node ੒Ԇ๢Ӥጱਂ‫ܫؙ‬ᬰᤈ඙֢ NodePublishVolume, NodeUnpublishVolume, NodeStageVolume. NodeUnstageVolume, NodeGetId, NodeGetInfo, NodeGetCapabilities …
12. ਂ‫ܫؙ‬ጱኞ޸ޮ๗ Controller ๐‫ۓ‬ CreateVolume ControllerPublishVolume Node ๐‫ۓ‬ NodeStageVolume NodePublishVolume CREATED NODE_READY VOL_READY PUBLISHED DeleteVolume ControllerUnpublishVolume NodeUnstageVolume NodeUnpublishVolume
13. ਂ‫ܫؙ‬အᶂ ਻࢏ଘ‫ݣ‬ CreateVolume Volume A Info CreateVolume Volume B Info CSI ൊկ Create Volume A Info Create Volume B Info ਂ‫ܫؙ‬ A အᶂѺ B
14. ଍ᒵ௔ ❖ ଍ᒵ௔ғֵአፘ‫݇ݶ‬හ੒຤ӻള‫᧣ݗ‬አग़ེӨ᧣አӞེጱᕮຎፘ‫̶ݶ‬ CreateVolume Create Volume A Info Volume A Info A ਻࢏ଘ‫ݣ‬ CreateVolume Volume A Info CSI ൊկ Find Volume A Info
15. ᥢ᝜ᬬࢧ‫؀‬ ❖ CSI ጱ CreateVolume ള‫ݗ‬Კ᧏Ꮁᥢ᝜ Condition Missing required field Invalid or unsupported field in the request Operation pending for volume Volume already exists but is incompatible Unable to provision in accessible_topology Unsupported capacity_range Call not implemented gRPC Code 3 INVALID_ARGUMENT 3 INVALID_ARGUMENT 10 ABORTED 6 ALREADY_EXISTS 8 RESOURCE_EXHAUSTED 11 OUT_OF_RANGE 12 UNIMPLEMENTED
16. • ᙧว • ୏‫ ݎ‬CSI ൊկ • ᮱ᗟ CSI ൊկ • ᨶᰁᓕቘ • ઀๕
17. Kubernetes ᮱ᗟ CSI ൊկ Kubernetes CSI വគ᮱ᗟොୗ https://github.com/kubernetes/community/blob/master/contributors/design-proposals/storage/containerstorage-interface.md ❖ ᭗‫מ‬ғֵአ UDS (Unix Domain Socket) ޾ HTTP ❖ ᮱ᗟғֵአ Kubernetes YAML ෈կ਻࢏۸᮱ᗟ ❖ ๦ᴴഴ‫ګ‬ғֵአ RBAC ᴴ‫ ګ‬CSI ൊկ๦ᴴ
18. ᭗‫מ‬ Controller ๐‫ۓ‬ Node ๐‫ۓ‬ Kubernetes ਂ‫ؙ‬ൊկ ਂ‫ؙ‬๐‫ۓ‬ API Server HTTP Kubernetes CSI UDS CSIਂ‫ؙ‬ൊկ HTTP ਂ‫ؙ‬๐‫ۓ‬ Kubelet UDS CSIਂ‫ؙ‬ൊկ HTTP ਂ‫ؙ‬๐‫ۓ‬ K8S ໐ஞᕟկ K8S CSI ੒ളᕟկ - Kubernetes ࢫᴚ CSI ൊկ - ਂ‫ࠟܯؙ‬
19. ᮱ᗟຝ຅ Master API Server Node1 Controller (StatefulSet) Kubelet External Provisioner External Attacher Node (DaemonSet) Driver Registrar CSI ਂ‫ؙ‬ൊկ CSI ਂ‫ؙ‬ൊկ Node2 Kubelet Node (DaemonSet) Driver Registrar CSI ਂ‫ؙ‬ൊկ Node3 Kubelet Node (DaemonSet) Driver Registrar CSI ਂ‫ؙ‬ൊկ … K8S ໐ஞᕟկ K8S CSI ੒ളᕟկ - Kubernetes ࢫᴚ CSI ൊկ - ਂ‫ࠟܯؙ‬
20. ᮱ᗟ YAML ෈կ ❖ ्กୗጱ Kubernetes YAML ਧ Ԏ෈կ҅‫׎‬ԭአಁࣁӧ‫ݶ‬ᵞᗭள ᭛޾ຽ‫ٵ‬۸ࣈ᮱ᗟൊկ̶
21. RBAC ๦ᴴഴ‫ګ‬ ❖ ਂ‫ؙ‬ൊկՐᵱ᧛ٟ Kubernetesᵞ ᗭጱ᮱‫ړ‬੒᨝҅ই: PV҅Nodeᒵ ੒᨝̶ ❖ ᭗ᬦ RBAC҅ᴴ‫ؙਂګ‬ൊկጱ๦ ᴴ҅ᖌಷᵞᗭਞ‫̶ق‬ kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1 metadata: name: csi-qingcloud-attacher labels: qingcloud.com/csi-controller: "true" csi: "v0.2.0" owner: "yunify" rules: - apiGroups: [""] resources: ["events"] verbs: ["get", "list", "watch", “update"] - apiGroups: [""] resources: ["persistentvolumes"] verbs: ["get", "list", "watch", "update"] - apiGroups: [""] resources: ["nodes"] verbs: ["get", "list", "watch"] - apiGroups: ["storage.k8s.io"] resources: ["volumeattachments"] verbs: ["get", "list", "watch", "update"]
22. • ᙧว • ୏‫ ݎ‬CSI ൊկ • ᮱ᗟ CSI ൊկ • ᨶᰁᓕቘ • ઀๕
23. ೮ᖅᵞ౮ ‫زܔ‬ၥᦶ ೮ᖅ᮱ᗟ (Kubernetes YAML file) ᵞ౮ၥᦶ (Kubernetes YAML file) ୏‫ݎ‬դᎱ ຅ୌ᳒‫؟‬ (Makefile, Kubernetes CSI Test
24. Kubernetes CSI Test ❖ CSI Test ฎ Kubernetes CSI ਥො୏‫ݎ‬ጱ୏რᶱፓ҅෰ࣁၥᦶ CSI ਂ‫ؙ‬ ൊկฎ‫ވ‬ᒧ‫ ݳ‬CSI ਥොᥢ᝜̶
25. • ᙧว • ୏‫ ݎ‬CSI ൊկ • ᮱ᗟ CSI ൊկ • ᨶᰁᓕቘ • ઀๕
26. ઀๕ ❖ ளᆙ‫ۑ‬ᚆਠ࠺ ❖ ਂ‫ܫؙ‬ፊഴ ❖ ਂ‫ܫؙ‬ಘ਻ ❖ In-tree ਂ‫ؙ‬ൊկ‫ ݻ‬CSI ᬢᑏ
27. ፘ‫ى‬ᩒრ CSI ਧԎᥢ᝜ғhttps://github.com/container-storage-interface/spec/blob/master/spec.md ᶆԯԯଘ‫ ݣ‬CSI ਂ‫ؙ‬ൊկғhttps://github.com/yunify/qingcloud-csi NeonSAN CSI ਂ‫ؙ‬ൊկғhttps://github.com/yunify/qingstor-csi Q&A