A Kubernetes Vagrant Multi node environment using kubeadm.
Switch branches/tags
Clone or download
galexrt Use kubeadm reset force flag (#21)
This prevents the cluster from starting up with current K8S versions (1.12>= it seems).
Otherwise the startup fails with `kubeadm reset` waiting for interactive response.
Latest commit 2cc840c Nov 23, 2018
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
data Initial commit Jun 7, 2018
.gitignore Initial commit Jun 7, 2018
CODE_OF_CONDUCT.md Initial commit Jun 7, 2018
LICENSE Initial commit Jun 7, 2018
Makefile Use built-in MAKE variable for make calls Sep 6, 2018
README.md Added KUBE_NETWORK variale to control CNI creation Jul 9, 2018
Vagrantfile Use kubeadm reset force flag Nov 23, 2018
Vagrantfile_nodes Use kubeadm reset force flag Nov 23, 2018

README.md

k8s-vagrant-multi-node

This project is based on work from coolsvap/kubeadm-vagrant by @coolsvap.

A demo of the start and destroy of a cluster can be found here: README.md Demo section.

Prerequisites

  • make
  • Vagrant (tested with 2.1.1)
  • Virtualbox
  • rsync
  • /dev/urandom (only used to generate a kubeadm token, when no custom KUBETOKEN is given)

Hardware Requirements

  • Master
    • CPU: 2 Cores
    • Memory: 2GB
  • 1x Node:
    • CPU: 1 Core
    • Memory: 1GB

These resources can currently only be changed by editing Vagrantfile and Vagrantfile_nodes directly, but will probably be configurable by environment variables in the future.

Quickstart

To start with the defaults, 1x master and 2x workers, run the following:

$ make up -j 3

The -j3 will cause three VMs to be started in parallel to speed up the cluster creation.

NOTE Your kubectl is automatically configured to use a context for the created cluster, after the master VM is started. The context is named after the directory the Makefile is in.

$ kubectl config current-context
k8s-vagrant-multi-node
$ kubectl get componentstatus
NAME                 STATUS    MESSAGE              ERROR
scheduler            Healthy   ok
controller-manager   Healthy   ok
etcd-0               Healthy   {"health": "true"}
$ kubectl get nodes
NAME      STATUS    ROLES     AGE       VERSION
master    Ready     master    4m        v1.10.4
node1     Ready     <none>    4m        v1.10.4
node2     Ready     <none>    4m        v1.10.4

Usage

Starting the environment

To start up the Vagrant Kubernetes multi node environment with the default of two worker nodes + a master (not parallel) run:

$ make up

NOTE Your kubectl is automatically configured to use a context for the created cluster, after the master VM is started. The context is named after the directory the Makefile is in.

Faster (parallel) environment start

To start up 4 VMs in parallel run (-j flag does not control how many (worker) VMs are started, the NODE_COUNT variable is used for that):

$ NODE_COUNT=3 make up -j4

The flag -j CORES/THREADS allows yout to set how many VMs (Makefile targets) will be run at the same time. You can also use -j $(nproc) to start as many VMs as cores/threads you have in your machine. So to start up all VMs (master and three nodes) in parallel, you would add one to the chosen NODE_COUNT.

Show status of VMs

$ make status
master                    not created (virtualbox)
node1                     not created (virtualbox)
node2                     not created (virtualbox)

Shutting down the environment

To destroy the Vagrant environment run:

$ make clean
$ make clean-data

Copy local Docker image into VMs

The make load-image target can be used to copy a docker image from your local docker daemon to all the VMs in your cluster. The IMG variable can be expressed in a few ways, for example:

$ make load-image IMG=your_name/your_image_name:your_tag
$ make load-image IMG=your_name/your_image_name
$ make load-image IMG=my-private-registry.com/your_name/your_image_name:your_tag

You can also specify a new image name and tag to use after the image has been copied to the VM's by setting the TAG variable. This will not change the image/tag in your local docker daemon, it will only affect the image in the VM's.

$ make load-image IMG=repo/image:tag TAG=new_repo/new_image:new_tag

Data inside VM

See the data/VM_NAME/ directories, where VM_NAME is for example master.

Show make targets

$ make help
clean                          Destroy master and node VMs, and delete data.
clean-data                     Remove data (shared folders) and disks of all VMs (master and nodes).
clean-master                   Remove the master VM.
clean-node-%                   Remove a node VM, where `%` is the number of the node.
clean-nodes                    Remove all node VMs.
help                           Show this help menu.
kubectl                        Configure kubeconfig context for the cluster using `kubectl config` (automatically done by `up` target).
load-image                     Load local/pulled Docker image into master and all node VMs.
load-image-master              Load local/pulled image into master VM.
load-image-node-%              Load local/pulled image into node VM, where `%` is the number of the node.
load-image-nodes               Load local/pulled Docker image into all node VMs.
preflight                      Run checks and gather variables, used for the the `up` target.
ssh-master                     SSH into the master VM.
ssh-node-%                     SSH into a node VM, where `%` is the number of the node.
start-master                   Start up master VM (automatically done by `up` target).
start-nodes                    Create and start all node VMs by utilizing the `node-X` target (automatically done by `up` target).
start-node-%                   Start node VM, where `%` is the number of the node.
status-master                  Show status of the master VM.
status-node-%                  Show status of a node VM, where `%` is the number of the node.
status-nodes                   Show status of all node VMs.
status                         Show status of master and all node VMs.
stop-master                    Stop/Halt the master VM.
stop-nodes                     Stop/Halt all node VMs.
stop-node-%                    Stop/Halt a node VM, where `%` is the number of the node.
stop                           Stop/Halt master and all nodes VMs.
token                          Generate a kubeadm join token, if needed (token file is `DIRECTORY_OF_MAKEFILE/.vagrant/KUBETOKEN`).
up                             Start Kubernetes Vagrant multi-node cluster. Creates, starts and bootsup the master and node VMs.

Variables

Variable Name Default Value Description
BOX_IMAGE generic/fedora27 Set the VMs box image to use.
DISK_COUNT 1 Set how many additional disks will be added to the VMs.
DISK_SIZE_GB 10 GB Size of additional disks added to the VMs.
MASTER_CPUS 1 Core Amount of cores to use for the master VM.
MASTER_MEMORY_SIZE_GB 2 GB Size of memory (in GB) to be allocated for the master VM.
NODE_CPUS 1 Amount of cores to use for each node VM.
NODE_MEMORY_SIZE_GB 1 GB Size of memory (in GB) to be allocated for each node VM.
NODE_COUNT 2 How many worker nodes should be spawned.
MASTER_IP 192.168.26.10 The Kubernetes master node IP.
NODE_IP_NW 192.168.26. The first three parts of the IPs used for the nodes.
POD_NW_CIDR 10.244.0.0/16 The Pod (container) network CIDR.
K8S_DASHBOARD false Install the Kubernetes dashboard addon.
CLUSTER_NAME k8s-vagrant-multi-node The name of the directory the Makefile is in.
KUBETOKEN "" (empty) The kubeadm "join" token to use. Will be generated automatically using /dev/urandom/ when empty.
KUBEADM_INIT_FLAGS "" (empty) The kubeadm init flags to use.
KUBEADM_JOIN_FLAGS "" (empty) The kubeadm join flags to use.
KUBERNETES_VERSION "" (empty) The kubeadm and kubelet package and API server version to install (KUBEADM_INIT_FLAGS will be set to --kubernetes-version=$KUBERNETES_VERSION if unset).
KUBE_PROXY_IPVS false Enable IPVS kernel modules to then use IPVS for the kube-proxy.
KUBE_NETWORK flannel What CNI to install, if empty don't install any CNI.

Demo

Start Cluster

asciicast

Destroy Cluster

asciicast