KubeCon 2018 NA SIG AWS Update

1. SIG-AWS Status & Roadmap @d-nishi, Nishi Davidson, AWS Container Services OSS SDM @justinsb, Justin SB, Google SDE
2. Summary • Goals: ○ Kubernetes integrations, interfaces, libraries, tools for all AWS services (IAM, storage, networking, loadbalancers, registry, security, monitoring/logging at the instance/container level) ○ Prow, Testgrid integrations for CI signal ○ Perf Dashboard for Scale testing signal ○ User group support for issues and feature requests ○ Documentation for all things Kubernetes on AWS • Hosts 5 subprojects: https://github.com/kubernetes/community/tree/master/sig-aws#subprojects • SIG Chairs: @d-nishi; @justinsb; @kris-nova • SIG AWS Charter: https://github.com/kubernetes/community/blob/master/sig-aws/charter.md
3. Subprojects launched in Kubernetes 1.13
4. aws-alb-ingress-controller - Status https://github.com/kubernetes-sigs/aws-alb-ingress-controller What we did: • • • • • Alpha released in Q4 ’18, k8s v 1.13. Stabilized support for IP target mode (routes traffic directly from ALB to pod IP without Nodeport hop). Re-factored code to use controller-runtime library and to add aws sdk level caching. Added e2e test cases. Add CI signal under https://testgrid.k8s.io/ Added more docs about setup/configure ingress controller. What we plan to do: • Add support to share one ALB across ingresses in different namespaces (Q1 ‘19) • Propose adding the controller in out-of-tree ccm Contact: @M00nF1sh, @bigkraig, @dnishi
5. Kubernetes Ingress • Sits in front of multiple services. • Acts as a “smart router” or entry point for http(s) traffic into the cluster • Many implementations: ALB, Nginx, F5, HAProxy etc • Default Service Type: ClusterIP
6. aws-alb-ingress-controller - Design • Runs on the worker nodes. • Enables host/path based routing, TLS termination, WebSockets, HTTP/2, AWS WAF integration, access logs, health checks • Supports two traffic modes: • instance mode: Traffic starts from the ALB, reaches the NodePort for the service and is then routed to the Pods. • ip mode: Traffic starts from the ALB and reaches the Pods directly. Used with AWS CNI plugin for Kubernetes
7. aws-alb-ingress-controller - Demo Demo
8. aws-ebs-csi-driver - Status https://github.com/kubernetes-sigs/aws-ebs-csi-driver What we did: • Alpha released in Q4 ’18, k8s v 1.13. Compatible with CSI 0.3.0 spec • Actively developing features to reach parity with in-tree volume controller design ○ Added storage class parameters: fsType, volume type, encrypted volume, etc ○ Added volume scheduling support ○ CI signal is reported under: https://testgrid.k8s.io/sig-aws-ebs-csi-driver ○ Next step: basic integration testing (only the driver on EC2 instance) What we plan to do: • CSI Migration for AWS EBS Driver in-progress. Targeting Q4 ‘19 to complete the work. Contact: @leakingtapan, @bertinatto, @d-nishi, @jsafrane
9. aws-ebs-csi-driver - Design Runtime Deployment of the driver
10. aws-ebs-csi-driver - Demo Demo
11. external cloudprovider-aws - Status In-tree CCM: • Add e2e tests for aws. • Maintain until out-of-tree ccm is GA (Q3 ‘19). • Deprecation period of 2 releases planned (Q3 ‘19). Out-of-tree CCM: • Alpha released in Q4 ’18, k8s v 1.13. • Extract cloud-provider-aws from k/k • Move cloudprovider dependencies from k/k to k/utils and staging • Scope testing for out-of-tree ccm. • Integrate CI signal in testgrid https://testgrid.k8s.io/ Contact: @mcrute, @dnishi
12. external cloudprovider-aws Demo
13. aws-k8s-tester - Status What we did: • Implements kubetest deployer interface (k8s.io/test-infra/kubetest) • Creates an ephemeral EKS cluster to run Kubernetes e2e tests as periodic jobs • Will be used for SIG AWS subproject CI signal and e2e tests What we will do: • Implement etcd conformance tests with Kubernetes • Integrate with cluster-api work to standardize testing Contact: @gyuho, @d-nishi
14. SIG Process Improvements • Subproject Election through formal KEPS: https://github.com/kubernetes/community/tree/master/keps/sig-aws • Subproject election process through lazy consensus followed by formal PR [Contact: @d-nishi] • How can you get involved? • Formal Issue tracking and closure by AWS engineers: https://github.com/orgs/K8s-AWSIssues/teams/github-issues • • Pick issues labeled “Easy” or “Medium” to get started. Members & Approvers from AWS now include: • @crutem, @M00nf1sh, @gyuho, @shyamjvs, @leakingthepan, @d-nishi, @nckturner, @mattlandis
15. Other projects
16. aws-encryption-provider - Status https://github.com/kubernetes-sigs/aws-encryption-provider Status: Existing subproject • Alpha in k8s v1.10. Requires etcd v3 • Integrates AWS-KMS-plugin with KMS provider on k8s control plane. KMS plugin (gRPC server) DEK • Enables envelope encryption of data in etcd. • Stores & manages config. data, auth. credentials independently. • Design and Scale discussions pending AWS KMS teams. • Testing and Documentation integration with upstream pending. • ASK: Use cases, User experience fb and Issues needed from users Remote KMS provider KEK
17. aws-iam-authenticator - Status https://github.com/kubernetes-sigs/aws-iam-authenticator Status: Existing subproject • Beta/GA in k8s v1.14. • Uses AWS IAM credentials to authenticate to a Kubernetes cluster. • Modifying current configmap implementation to CRD is pending. • Testing and Documentation integration with upstream pending.
18. cluster-api-aws – Status https://github.com/kubernetes-sigs/cluster-api-provider-aws Status: Co-owned subproject with SIG Cluster Lifecycle (@ • Alpha in k8s v1.13. • Standardizes bootstrapping of a Kubernetes cluster using best practices from KOPS, KUBICORN • Bootstraps VPCs, gateways, security groups and instances. • Supports Amazon Linux 2, CentOS 7 and Ubuntu 18.04, using pre-baked AMIs. • Deploys Kubernetes control planes into private subnets with a separate bastion server. • Doesn't use SSH to bootstrap nodes. • Installs minimal components to bootstrap a control plane and workers. • Currently supports control planes on EC2 instances. • ASK: Feedback on User experience and Issues from testing needed from users (also for EKS)
19. eksctl - Status https://github.com/weaveworks/eksctl Status: Proposed Subproject (@errordeveloper) • CLI tool to create Amazon EKS clusters. • Written in Go, and uses CloudFormation. • Creates a cluster in minutes with one command – eksctl create cluster
20. etcd - Status https://github.com/etcd-io/etcd Status: Related project. CNCF Incubating project actively maintained by AWS (@gyuho) • Future roadmap of etcd from the maintainers: <blog link> • Summary of roadmap updates ○ Support non-voting member, improve membership reconfiguration ○ Support downgrade ○ Improve conformance testing with Kubernetes control plane
21. 2019 Plan as of 11/12/2018 *subject to changes Q4 ‘18 v1.13 1. SERVICES: aws-alb-ingress-controller ALPHA 1. STORAGE: aws-ebs-csi-driver ALPHA 1. CLOUDPROVIDER-AWS: Out-of-tree ALPHA Q1 ‘19 v1.14 1. SERVICES: aws-alb-ingress-controller BETA ++ more features nlb in in-tree BETA 1. STORAGE: aws-ebs-csi-driver BETA aws-efs-csi-driver ALPHA In-tree volume plugin migration 1. CLOUDPROVIDER-AWS: In-tree provider deprecation start Out-of-tree provider BETA v1.15 1. SERVICES: aws-alb-ingress-controller GA nlb in-tree k8s GA 2. STORAGE: aws-ebs-csi-driver GA aws-efs-csi-driver BETA In-tree volume plugin MIGRATION 2. CLOUDPROVIDER-AWS: In-tree provider DEPRECATION END Out-of-tree provider GA 2. ADVANCE OTHER PROJECTS 2. ADVANCE OTHER PROJECTS Q3 ‘19 Q2 ‘19 v1.16