Unless otherwise specified... /* Copyright 2017 The Kubernetes Authors. Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. */
Why Are We Copying and Pasting So Much? Philip Wittrock (@pwittrock) & Solly Ross (@directxman12), Google
Kubernetes APIs are implemented by Controllers ● Respond to changes to Resources made by users ● Respond to changes to Cluster State
Controllers Are Simple ✓ ✓ ✓ ✓ watch resources, do some business logic, reconcile differences, … and a bit of plumbing How complicated could it be?
Sample Controller at a Glance ~/go/src/k8s.io/sample-controller $ wc -l < controller.go 429 ~/go/src/k8s.io/sample-controller $ tree -I vendor <snip> 28 directories, 52 files
This is not a flame chart Note: Boilerplate code to copy-paste highlighted in Red
At least the comments are thorough... // // // // // // … // // // // // … // // // We call Done here so the workqueue knows we have finished processing this item. We also must remember to call Forget if we do not want this work item being re-queued. For example, we do not call Forget if a transient error occurs, instead the item is put back on the workqueue and attempted again after a back-off period. We expect strings to come off the workqueue. These are of the form namespace/name. We do this as the delayed nature of the workqueue means the items in the informer cache may actually be more up to date that when the item was initially put onto the workqueue. As the item in the workqueue is actually invalid, we call Forget here else we'd go into a loop of attempting to process a work item that is invalid.
Controllers are Simple? 5. ien we Cl s te s a ra ter ne Lis Ge . a 4. Be Re gin a. con Pop c p b. Er ile ing Copy Clients to Reconciler Struct c. fro Us ror m Create a Queue for Reconcile Requests (namespace/name) Fo e R Ha Qu n r a d Add EventHandlers for all Object Types that you are interested in eu i. get teli ling e th mi a. Don’t Enqueue for Deletion Events on the Reconciled Type an Un e O tin Fo r d g l i. Unless you have a finalizer, then you should Re er es bj Ca w e ro s h c c llin o e b. Do Enqueue for Deletion Events on Owned Types r - th t fr n n g c e o th re m Re ile i. But find the owner of the object to Reconcile instead en i q t do s a he Q ueu 1. Write Handlers to walk owners References to find parents n’ tra u ing a. Be sure to handle Tombstones correctly t F n eu or sie e i. Actually maybe we don’t need to do that anymore? ge nt t c. Write Logic to Turn Objects Into Keys i. Type Cast Request to String 9. ct Into ame e j 1. Error Handling if this fails d Se Ob ring e / N wne er e t Start Worker Threads t cil lit S pac uno own a. up S n o a. But not too many c Sp mes for ing ign b. e U R . a n Start Informers Na eck s us W ix l Ha a . ind a. Wait For Cache To Sync Ch ject nce nd 8 o b e l . 10 1. 2. 3. 7.
Which is the real Sample Controller? Sample Controler + Core Kubernetes Controller + Third Party Operator
Controllers Should Be Simple 65% of the sample controller code is… ● Unmodified Copy-Pasted Functions ● Initialization and Instantiation of Deps ● Plumbing and Wiring
Native Controller Support in Go 1.12! controller-runtime (proper noun) a Go standard library for controllers package main import ( "controllers" ) Just Kidding
Seriously though, use this instead controller-runtime (proper noun) a Go standard library for controllers package main import ( "sigs.k8s.io/controller-runtime/pkg/builder" )
The magic of abstractions... Patterns ● Watch Objects to Reconcile ● Watch Generated Objects ● Watch Objects that Map to Reconciled Objects Utilities ● Pre-Split Namespace/Name ● Universal Client (no more wiring generated clients) ● Helper for setting Owners References ● Helper for Upserting Objects
Sophisticated Watch APIs The.ForType builder function is really just a call to .Watch with EnqueueRequestForObject
Sophisticated Watch APIs The.Owns builder function is really just a call to .Watch with EnqueueRequestForOwner
Sophisticated Watch APIs Users can implement their own patterns by providing a function to map events to objects
For Ops... Standard… ● Behavior ● Logging ● Metrics ● Error handling and degradation ● Maintenance
For Platform Developers... Build high-level abstractions with common building blocks: ● ● ● ● Kubebuilder Operator SDK Maestro Declarative Operator (via kubebuilder) AddOnOperator (via kubebuilder)
In conclusion... Controllers Are Simple package main import ( "sigs.k8s.io/controller-runtime/pkg/builder" )