Skip to content
Kubernetes Extended Daemonset controller
Go Shell Other
Branch: master
Clone or download
Latest commit 5163730 Nov 19, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.github first commit Nov 19, 2019
build first commit Nov 19, 2019
chart/extendeddaemonset first commit Nov 19, 2019
cmd first commit Nov 19, 2019
deploy
examples first commit Nov 19, 2019
hack
pkg first commit Nov 19, 2019
test
vendor
version first commit Nov 19, 2019
.dockerignore
.envrc first commit Nov 19, 2019
.gitignore first commit Nov 19, 2019
.gitlab-ci.yml first commit Nov 19, 2019
.golangci.toml first commit Nov 19, 2019
Dockerfile first commit Nov 19, 2019
LICENSE
LICENSE-3rdparty.csv first commit Nov 19, 2019
Makefile first commit Nov 19, 2019
README.md
go.mod first commit Nov 19, 2019
go.sum
go.test.sh first commit Nov 19, 2019
tools.go

README.md

ExtendedDaemonSet

!!! Alpha version !!!

ExtendedDaemonSet aims to provide a new implementation of the Kubernetes DaemonSet resource with key features:

  • Canary Deployment: Deploy a new DaemonSet version with only a few nodes.
  • Custom Rolling Update: Improve the default rolling update logic available in Kubernetes batch/v1 Daemonset.

How to use it

Deployment

To use the ExtendedDaemonSet controller in your Kubernetes cluster, deploy its resources definitions in the /deploy folder.

First, deploy the Custom Resources Definitions:

$ kubectl apply -f ./deploy/crds

If you want to deploy a specific controller version, the version can be set in the deploy/operator.yaml file. By default, the latest docker image tag is used.

$ kubectl apply -f ./deploy/

By default, the controller only watches the ExtendedDaemonSet resources that are present in its own namespace. If you want to deploy the controller cluster wide, modify the deploy/operator.yaml to remove or set an empty value for the WATCH_NAMESPACE environment variable:

            env:  
            - name: WATCH_NAMESPACE
              value: ""

Demo application

If you want to test and compare the advantages of the ExtendedDaemonSet over the the standard DaemonSet, you can use the demo application available in the /example folder. Follow the below scenario:

First, you need a Kubernetes cluster with several nodes; we recommend using three nodes. If you want, you can use kind.sigs.k8s.io to create a three node cluster with the following command: kind create cluster --config examples/kind-cluster-configuration.yaml.

This creates a three node cluster with one master and two worker nodes:

$ kind create cluster --config examples/kind-cluster-configuration.yaml
Creating cluster "kind" ...
 ✓ Ensuring node image (kindest/node:v1.15.3) 🖼
 ✓ Preparing nodes 📦📦📦 
 ✓ Creating kubeadm config 📜 
 ✓ Starting control-plane 🕹️ 
 ✓ Installing CNI 🔌 
 ✓ Installing StorageClass 💾 
 ✓ Joining worker nodes 🚜 
Cluster creation complete. You can now use the cluster with:

$ export KUBECONFIG="$(kind get kubeconfig-path --name="kind")"

ExtendedDaemonSet controller deployment

# deploy the controller needed crds
$ kubectl apply -f ./deploy/crds
customresourcedefinition.apiextensions.k8s.io/extendeddaemonsets.datadoghq.com created
customresourcedefinition.apiextensions.k8s.io/extendeddaemonsetreplicasets.datadoghq.com created

# deploy the controller pod
$ kubectl apply -f ./deploy/
clusterrole.rbac.authorization.k8s.io/extendeddaemonset created
clusterrolebinding.rbac.authorization.k8s.io/extendeddaemonset created
deployment.apps/extendeddaemonset created
role.rbac.authorization.k8s.io/extendeddaemonset created
rolebinding.rbac.authorization.k8s.io/extendeddaemonset created
serviceaccount/extendeddaemonset created

# you should see the extendeddaemonset controller pod running
$ kubectl get pods
NAME                                 READY   STATUS    RESTARTS   AGE
extendeddaemonset-855cd7c679-gpmql   1/1     Running   0          2m11s

foo ExtendedDaemonSet deployment

Create the foo app with the ExtendedDaemonSet. For demo purposes, we'll use the k8s.gcr.io/pause Docker image, which is only awaiting a terminating signal. You can look at the foo application definition in the file examples/foo-eds_v1.yaml.

$ kubectl apply -f examples/foo-eds_v1.yaml
extendeddaemonset.datadoghq.com/foo created

You can see the state of the ExtendedDaemonSet foo with:

$ kubectl get eds
NAME   DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   STATUS    ACTIVE RS   CANARY RS   AGE
foo    3         3         3       3            3           Running   foo-8z7lr               44s

# Also the `extendeddaemonsetreplicaset` resource generated by the controller from the `foo` EDS instance:
$ kubectl get ers
NAME        STATUS   DESIRED   CURRENT   READY   AVAILABLE   NODE SELECTOR   AGE
foo-8z7lr   active   3         3         3       3                           61s

foo ExtendedDaemonSet deployment update with canary strategy

Now we can try to update the ExtendedDaemonSet foo. The only difference between the two versions is the Docker image used in the pod template.

$ diff examples/foo-eds_v1.yaml examples/foo-eds_v2.yaml
17c17
<         image: k8s.gcr.io/pause:3.0
---
>         image: k8s.gcr.io/pause:3.1

$ kubectl apply -f examples/foo-eds_v2.yaml
extendeddaemonset.datadoghq.com/foo configured

As you can see with the following command, a canary ReplicaSet is now configured for the foo ExtendedDaemonSet. Additionally, a new ExtendedReplicaSet has been created to handle the new foo pod template version.

$ kubectl get eds
NAME   DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   STATUS   ACTIVE RS   CANARY RS   AGE
foo    3         3         3       3            3           Canary   foo-8z7lr   foo-xdj4b   85s

$ kubectl get ers
NAME        STATUS   DESIRED   CURRENT   READY   AVAILABLE   NODE SELECTOR   AGE
foo-8z7lr   active   2         2         2       2                           2m
foo-xdj4b   canary   1         1         1       1                           40s

$ kubectl get pod -l extendeddaemonset.datadoghq.com/name=foo
NAME                                 READY   STATUS    RESTARTS   AGE
foo-8z7lr-bp9w8                      1/1     Running   0          108s
foo-8z7lr-jlvrq                      1/1     Running   0          88s
foo-xdj4b-zvss2                      1/1     Running   0          8s

Only one pod is running with the ExtendedReplicaSet foo-xdj4b pod template version. This corresponds to the setting spec.canary.replicas in the ExtendedDaemonSet foo.

Rolling update after the canary deployment validation period ended

After 5 minutes, which corresponds to spec.canary.duration, the controller will set as valid and activate the foo-xdj4b ExtendedReplicaSet. It will trigger the full foo-xdj4b ExtendedReplicaSet deployment.

$ kubectl get eds
NAME   DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   STATUS    ACTIVE RS   CANARY RS   AGE
foo    3         3         3       3            3           Running   foo-xdj4b               9m21s

$ kubectl get ers
NAME        STATUS   DESIRED   CURRENT   READY   AVAILABLE   NODE SELECTOR   AGE
foo-xdj4b   active   3         3         3       3                           8m21s

$ kubectl get pod -l extendeddaemonset.datadoghq.com/name=foo
NAME              READY   STATUS    RESTARTS   AGE
foo-xdj4b-hh6d8   1/1     Running   0          5m11s
foo-xdj4b-rgtk9   1/1     Running   0          5m31s
foo-xdj4b-zvss2   1/1     Running   0          10m

Kubectl plugin

To build the the kubectl ExtendedDaemonSet plugin, you can run the command: make build-plugin. This will create the kubectl-eds Go binary, corresponding to your local OS and architecture. Then, add or move this binary to the PATH and run the command kubectl eds:

$ kubectl eds
Usage:
  ExtendedDaemonset [command]

Available Commands:
  get         get ExtendedDaemonSet deployment(s)
  get-ers     get-ers ExtendedDaemonSetReplicaset deployment(s)
  help        Help about any command
  validate    validate canary replicaset

How to migrate from a DaemonSet

If you already have an application running in your cluster with a DaemonSet, it is possible to migrate to an ExtendedDaemonSet with a smooth migration path.

  • Update your DaemonSet specification to set a toleration that does not correspond to your node's taints. As a result, the DaemonSet pods that are already running will not be deleted, and the DaemonSet controller will not take any new actions on it.

  • In the ExtendedDaemonSet definition, add a specific annotation to inform the extendeddaemonset-controller which DaemonSet needs to be migrated. The controller will recognize the pods from the "old" DaemonSet as a previous version and will do a proper rolling update.

apiVersion: datadoghq.com/v1alpha1
kind: ExtendedDaemonSet
metadata:
  name: foo
  annotations:
    extendeddaemonset.datadoghq.com/old-daemonset: foo
spec:
    # ...

Developers section

How to build it

This project uses go module. Ensure you have it activated: export GO111MODULE=on.

Run make install-tools to install mandatory tooling, like the operator-fwk command line and the golangci linter.

$ make build
CGO_ENABLED=0 go build -i -installsuffix cgo -ldflags '-w' -o controller ./cmd/manager/main.go

How to test it

Unit tests

$ make test
./go.test.sh
ok      github.com/datadog/extendeddaemonset/pkg/controller/extendeddaemonset   1.107s  coverage: 77.0% of statements
ok      github.com/datadog/extendeddaemonset/pkg/controller/extendeddaemonsetreplicaset 1.098s  coverage: 63.9% of statements
ok      github.com/datadog/extendeddaemonset/pkg/controller/extendeddaemonsetreplicaset/strategy        1.036s  coverage: 5.3% of statements
ok      github.com/datadog/extendeddaemonset/pkg/controller/extendeddaemonsetreplicaset/strategy/limits 1.016s  coverage: 83.3% of statements
ok      github.com/datadog/extendeddaemonset/pkg/controller/utils       1.015s  coverage: 100.0% of statements

End to end tests

For end to end tests, consult /test/README.md

Linter validation

To use the linter, run:

$ make validate
./bin/golangci-lint run ./...
You can’t perform that action at this time.