Demo: Run one or more dump1090-mutability instances on a Kubernetes cluster (in the cloud).
Horizontal scaling in Docker containers behind a load balancer makes dump1090 accessable for a large audiences.
Running a dump1090 in a Kubernetes cluster makes it possible to do:
- rolling updates of new versions of dump1090 without down time.
- roll back to a previous version instantly.
- improve availability.
- scale up and down the number of dump1090 instances depending on the number of load.
- Create a single pod with a dump1090 instance.
- Create a replication controller with multiple dump1090 instances.
- Create a deployment of multiple dump1090 instances.
- Create a load balancer service for multiple dump1090 instances with an external IP address.
- Scale up and scale down dump1090 instances.
- Do a rolling update to a new version of dump190.
- Roll back to a previous version.
If you are unfamiliar with Kubernetes you can watch this video https://www.youtube.com/watch?v=WwBdNXt6wO4 by Brendan Burns, Software engineer and a founder of Kubernetes at Google.
Or read the technical over view: http://kubernetes.io/docs/whatisk8s
If you don't like to set up a Kubernetes cluster yourself, you can use a 60 days/$300 free trail on Google Cloud: https://cloud.google.com
All you need is less than 5 minutes to signup and add your credit card details before you can start.
Note: After the free trail period your access to the cluster automatically ends without any payment.
Start with: https://cloud.google.com/container-engine/docs/before-you-begin or follow these brief instructions:
After selecting 'Container Engine', you can create a new 'Container Cluster'. Choose the defaults is just fine.
Launch a Google Cloud Shell from your Google Cloud Console (indicated with the green arrow in the Google Cloud Console screenshot below).
To perform this demo I did use the Google Cloud Shell to enter the cli commands. Google Cloud Shell is a Debian based OS with a 5GB home directory and all the necessary tools (like gcloud, kubectl, docker, git, etc) already installed.
Tell the command-line interface which Google Cloud Platform project to use:
gcloud config set project YOUR_GOOGLE_CLOUD_PROJECT
gcloud config set compute/zone YOUR_GOOGLE_COMPUTE_ZONE
gcloud config set container/cluster YOUR_KUBERNETES_CLUSTER_NAME
gcloud container clusters get-credentials YOUR_KUBERNETES_CLUSTER_NAME
gcloud config list
You can find the correct settings in your own Google Cloud Console (as indicated with the red arrows in the Google Cloud Console screenshot above).
Download the yaml files to your Google Cloud Shell where you can run kubectl.
wget https://raw.githubusercontent.com/tedsluis/dump1090-kubernetes/master/dump1090-pod.yaml
wget https://raw.githubusercontent.com/tedsluis/dump1090-kubernetes/master/dump1090-replication-controller.yaml
wget https://raw.githubusercontent.com/tedsluis/dump1090-kubernetes/master/dump1090-deployment-v1.yaml
wget https://raw.githubusercontent.com/tedsluis/dump1090-kubernetes/master/dump1090-deployment-v2.yaml
wget https://raw.githubusercontent.com/tedsluis/dump1090-kubernetes/master/dump1090-services.yaml
Or clone the repo and move to the repo directory:
git clone https://github.com/tedsluis/dump1090-kubernetes.git
cd dump1090-kubernetes
A Pod is the basis scheduling unit within Kubernetes: One or more Docker containers that run on the same Kubernetes node that share resources. http://kubernetes.io/docs/user-guide/pods
Proof that there are no running pods at the start of this demo:
kubectl get pods
Create a single pod with one dump1090 container:
kubectl create -f dump1090-pod.yaml
List running pods:
kubectl get pods
List details pod:
kubectl describe pods
Kubernetes provides service discovery and deals with routing by assigning a stable IP address and DNS name. It provides round-robin load balancing of network connections to the pods matching the selector (even as failures cause the pods move to another Kubernetes node). A service is exposed inside a cluster, but a service can also be exposed outside a cluster. http://kubernetes.io/docs/user-guide/services
This services creates a load balancer for the dump1090 web server(s). It also provides a external IP address.
Check to running services:
kubectl get services
Only the Kubernetes cluster service is running. Leave that as is.
Create the service:
kubectl create -f dump1090-services.yaml
Check the service:
kubectl get services
Note: It will take a minute before you get an external IP address!
After a minute:
kubectl get services
Do you see external IP address for dump1090? Use that in your internet browser!
Show details services:
kubectl describe services dump1090
Check dump1090 in your browser. Use the 'EXTERNAL IP' that was displayed by the 'kubectl get service' command.
You may need to refresh your browser a view times!
http://external_ip_address_of_service/dump1090
Note: Everyone on the internet can use this dump1090 now!
You can try my setup: http://104.197.175.34/dump1090/gmap.html
For the next example we first have to delete this our pod:
kubectl delete pods dump1090
note: We leave the service with the external IP address as it is for the next examples.
List pods:
kubectl get pods
A replication controller is a set of Pods that perform the same function. A replication controller provides scaling (by changing the number of Pods) and availability since the replication controller always replaces a Pod when one fails. http://kubernetes.io/docs/user-guide/replication-controller
Create a replication controller that exists of 3 Pods:
kubectl create -f dump1090-replication-controller.yaml
List the Pods:
kubectl get pods
Scale up the replication controller to 5 dump1090 instances:
kubectl scale --replicas=5 rc dump1090
You can scale down the number of pods in the same way.
List the replication controller:
kubectl get rc
Every pod has the dump1090 label:
kubectl get pods --selector=name=dump1090
Kubernetes uses labels and selectors to group resources. http://kubernetes.io/docs/user-guide/labels
This 'dump1090' label is used by the load balancer to identify the dump1090 pods. That label was attached to the pods and service in the yaml configuration files.
Note that the 'dump1090' service is still active.
All pods are accessable using the external IP address (port 80) dune to the label 'dump1090'.
The service will balances the load over the pods:
kubectl get services
Check dump1090 in your browser. You may need to refresh your browser a view times!
http://external_ip_address_of_service/dump1090
Remove the replication controller before you start the next demo example:
kubectl delete rc dump1090
The pods are gone:
kubectl get pods
All pods are removed, so we are ready for the next demo.
Deployment describes the desired state of Pods and replication sets in Kubernetes. Typical use cases are rolling updates and down grades. http://kubernetes.io/docs/user-guide/deployments
This deployment starts 3 pods with dump1090.
kubectl create -f dump1090-deployment-v1.yaml
Note: 'Deployment' is still a beta feature at the time I am testing this. You may get an error. You may need to change the API version within the yaml file. Check the kubernetes resources http://kubernetes.io/docs/user-guide/deployments/ for more info in case you encounter any trouble.
Check the deployment:
kubectl get pods
List Deployments:
kubectl get deployments
Show detail Deployment:
kubectl describe deployments
Check dump1090 in your browser. You may need to refresh your browser a view times!
http://external_ip_address_of_service/dump1090
This deployment replaces 3 pods with an other version of dump1090.
kubectl apply -f dump1090-deployment-v2.yaml
Check the deployment:
kubectl get pods
The version 2 pods are running and the version 1 pods are terminated!
kubectl get deployments
Check dump1090 in your browser. You may need to refresh your browser a view times!
http://external_ip_address_of_service/dump1090
This deployment replaces 3 pods with version 1 of dump1090.
kubectl apply -f dump1090-deployment-v1.yaml
kubectl get pods
The version 1 pods are running and the version 2 pods are terminated!
Check dump1090 in your browser. You may need to refresh your browser a view times!
http://external_ip_address_of_service/dump1090
You have seen how easy it is to host a scaled dump1090 application in a Kubernetes cluster.
Used docker container images:
https://hub.docker.com/r/tedsluis/dump1090-mutability/ (version v1.15_heatmaprangeview and v1.15)
Github sources:
https://github.com/tedsluis/docker-dump1090
https://github.com/tedsluis/dump1090
https://github.com/mutability/dump1090
Try my dump1090 fork with heatmap and rangeview in the Google cloud: http://104.197.44.48/dump1090/gmap.html (This dump1090 runs on a 60-day free trail that is available until 20 june 2016, more info at https://cloud.google.com/free-trial/)
Ted Sluis
ted.sluis@gmail.com
https://github.com/tedsluis
https://hub.docker.com/r/tedsluis
https://www.youtube.com/tedsluis
http://flightaware.com/adsb/stats/user/tedsluis