This controller shifts nodes from one node pool to another, in order to favour preemptibles over a 'safety net' node pool of regular vms
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
chart/estafette-gke-node-pool-shifter
vendor
.estafette.yaml
.gitignore
Dockerfile
LICENSE
README.md
gcloud.go
gcloudContainer.go
govendor.sh
helpers.go
helpers_test.go
kubernetes.go
kubernetes.yaml
main.go
rbac.yaml

README.md

estafette-gke-node-pool-shifter

This controller shifts nodes from one node pool to another, in order to favour preemptibles over a 'safety net' node pool of regular vms.

License

Usage

You can either use environment variables or flags to configure the following settings:

Environment variable Flag Default Description
INTERVAL --interval (-i) 300 Time in second to wait between each shift check
KUBECONFIG --kubeconfig Provide the path to the kube config path, usually located in ~/.kube/config. For out of cluster execution
METRICS_LISTEN_ADDRESS --metrics-listen-address :9001 The address to listen on for Prometheus metrics requests
METRICS_PATH --metrics-path /metrics The path to listen for Prometheus metrics requests
NODE_POOL_FROM --node-pool-from Name of the node pool to shift from
NODE_POOL_FROM_MIN_NODE --node-pool-from-min-node 0 Minimum amount of node to keep on the from node pool
NODE_POOL_TO --node-pool-to Name of the node pool to shift to

Before deploying, you first need to create a service account via the GCloud dashboard with role set to Compute Instance Admin and Kubernetes Engine Admin. This key is going to be used to authenticate from the application to the GCloud API. See documentation.

Deploy with Helm

# Prepare Helm/Tiller
$ kubectl create sa tiller -n kube-system
$ helm init --service-account tiller
$ kubectl create clusterrolebinding tiller \
    --clusterrole=cluster-admin \
    --serviceaccount=kube-system:tiller

# Install
$ helm upgrade estafette-gke-node-pool-shifter \
    --namespace estafette \
    --install \
    --set rbac.create=true \
    --set-file googleServiceAccount=./google_service_account.json \
    --set nodePool.from=default-pool \
    --set nodePool.to=preemptible-pool \
    chart/estafette-gke-node-pool-shifter

Deploy without Helm

export NAMESPACE=estafette
export APP_NAME=estafette-gke-node-pool-shifter
export TEAM_NAME=tooling
export VERSION=1.0.11
export GO_PIPELINE_LABEL=1.0.11
export GOOGLE_SERVICE_ACCOUNT=$(cat google-service-account.json | base64)
export INTERVAL=300
export NODE_POOL_FROM=default-pool
export NODE_POOL_TO=preemptible-pool
export NODE_POOL_FROM_MIN_NODE=0
export CPU_REQUEST=10m
export MEMORY_REQUEST=16Mi
export CPU_LIMIT=50m
export MEMORY_LIMIT=128Mi

# Setup RBAC
curl https://raw.githubusercontent.com/estafette/estafette-gke-node-pool-shifter/master/rbac.yaml | envsubst | kubectl apply -n ${NAMESPACE} -f -

# Run application
curl https://raw.githubusercontent.com/estafette/estafette-gke-node-pool-shifter/master/kubernetes.yaml | envsubst | kubectl apply -n ${NAMESPACE} -f -

Local development

For development purpose, you can create a new cluster with 2 autoscaled node pools, 1 preemptible and 1 regular VM.

Create the cluster with appropriate node pools

export CLUSTER_NAME=node-shifter
export CLUSTER_VERSION=1.7.3
export PROJECT=my-project
export ZONE=europe-west1-c

# Create cluster with regular VMs
gcloud beta container clusters create $CLUSTER_NAME \
  --project=$PROJECT \
  --zone=$ZONE \
  --cluster-version=$CLUSTER_VERSION \
  --num-nodes=1 \
  --enable-autoscaling \
  --min-nodes=0 \
  --max-nodes=3

# Add preemptible VMs node pool
gcloud beta container node-pools create preemptible-pool \
  --project=$PROJECT \
  --zone=$ZONE \
  --cluster=$CLUSTER_NAME \
  --num-nodes=1  \
  --enable-autoscaling \
  --min-nodes=1 \
  --max-nodes=3 \
  --preemptible

Deploy an application

kubectl run nginx --image=nginx:alpine --replicas=5 --limits='cpu=200m,memory=512Mi'

Start the node pool shifter

# proxy master
kubectl proxy

# in another shell
go build && ./estafette-gke-node-pool-shifter --node-pool-from=default-pool --node-pool-to=preemptible-pool

Note: KUBECONFIG=~/.kube/config as environment variable can also be used if you don't want to use the kubectl proxy command.

If necessary, you can resize the node pool size:

gcloud container clusters resize $CLUSTER_NAME
  --project=$PROJECT \
  --zone=$ZONE \
  --size=1 \
  --node-pool=default-pool