Evaluate simpler kubeadm deploy

[plunder-app]

There are a number of deployment options for using a vip with kubeadm all of them have numerous pros/cons:

1. Standalone VIP pre-kubeadm
This is a "one-shot" approach where a vip is created and applied prior to running a kubeadm init, this will attach the IP to the initial host allowing kubeadm to communicate with the newly deployed api-server through the address. Once completed then this vip would need removing and restarting within the cluster itself (perhaps as a daemonset).

This could be automated in the following fashion:

• kube-vip is started and is passed the command line to run kubeadm
• kube-vip then runs the kubeadm init and waits for its completion exec.Command()
• kube-vip upon completion of the init stops the vip and starts it within the kubernetes cluster

2. Static pods (how it is done today)
In this approach the vip is ran outside of the cluster, through the use of static pods that are managed by the kubelet and not the kubernetes cluster itself.

This is automated in the following fashion:

• kube-vip generates a manifest in /etc/kubelet/manifests
• kubeadm init starts all pods from manifests (including kube-vip)
• kubeadm join followed by kube-vip join will add nodes through the vip and will add them as members of the vip cluster

cc @yastij / @ncdc / @randomvariable (any thoughts)

[ThomasLohmann]

Hi @plunder-app @thebsdbox, first of all, great job so far!

I have 3 general questions:

1. Is Kube-vip production-ready? I couldn't find any information about that

2. Does Kube-vip work with Kubespray (uses kubeadm) in the HA mode? If not, it would be great if we can find a way to integrate Kube-vip into Kubespray. Please, see the following link: https://kubespray.io/#/docs/ha-mode

3. Does Kube-vip has any disadvantages compared to traditional approaches like envoy + keepalived from the load balancer perspective?

Thanks in advance!

[thebsdbox]

Hello @ThomasLohmann,

1. Hard to answer really, It does what it says on the tin but more "production-ready" functionality could be added (monitoring or advanced logging etc..). Best option is to test it out and give feedback.

2. Not at this time, there is documentation for using it with kubeadm -> https://github.com/kubernetes/kubeadm/blob/master/docs/ha-considerations.md#kube-vip but I presume additional work would be required in order to have it automated.

3. Kube-vip was written in order to simplify the process of having multiple tooling and multiple configurations required just to provide load balancing capabilities to the control plane. So the hope is that it only has advantages :)

[randomvariable]

Potentially, a consumer of kube-vip may not execute kubeadm init and instead use phases. Ultimately, I guess you could check the /readyz status of API server and kubelet, and then you're partially agnostic of using kubeadm or not.

[thebsdbox]

I’ve modified kube-vip to “mirror” kubeadm join|init commands, which should make the usage a lot cleaner. I guess it will be in the 0.1.5 version once I’ve tidied things up a little bit, although it’s relatively stable at the moment (my demo is working)

Init process

With Docker

sudo docker run --network host --rm plndr/kube-vip:0.1.5 kubeadm init --interface ens192 --vip 192.168.0.81 --startAsLeader=true | sudo tee /etc/kubernetes/manifests/vip.yaml

Without Docker
sudo ./kube-vip kubeadm init --interface ens192 --vip 192.168.0.81 --startAsLeader=true | sudo tee /etc/kubernetes/manifests/vip.yaml

Init Kubernetes Cluster

sudo kubeadm init --kubernetes-version 1.17.0 --control-plane-endpoint 192.168.0.81 --upload-certs

Join Process

Join Kubernetes Cluster

sudo kubeadm join 192.168.0.81:6443 --token w5atsr.blahblahblah --control-plane --certificate-key abc123

With Docker

sudo docker run -v /etc/kubernetes/admin.conf:/etc/kubernetes/admin.conf --network host --rm plndr/kube-vip:0.1.5 kubeadm join --interface ens192 --vip 192.168.0.81 --startAsLeader=false | sudo tee /etc/kubernetes/manifests/vip.yaml

Without Docker

sudo ./kube-vip kubeadm join --interface ens192 --vip 192.168.0.81 --startAsLeader=false | sudo tee /etc/kubernetes/manifests/vip.yaml

[thebsdbox]

This has been merged and documentation has been updated (#46 )