kube-ansible is a set of Ansible playbooks and roles that allows
you to instantiate a vanilla Kubernetes cluster on (primarily) CentOS virtual
machines or baremetal.
Additionally, kube-ansible includes CNI pod networking (defaulting to Flannel, with an ability to deploy Weave and Multus).
The purpose of kube-ansible is to provide a simpler lab environment that allows prototyping and proof of concepts. For staging and production deployments, we recommend that you utilize OpenShift-Ansible
Playbooks are located in the
||Provision a virtual machine host|
||Install and configure a k8s cluster|
||Destroys VMs on the virtual machine host|
||Install a GlusterFS cluster across VMs (requires vm-attach-disk)|
||Bootstrapping Python dependencies on cloud images|
|builder.yml||./inventory/vms.local.generated||Build a Kubernetes release in a dedicated virtual machine|
(Table generated with markdown tables)
kube-ansible provides the means to install and setup KVM as a virtual host platform on which virtual machines can be created, and used as the foundation of a Kubernetes cluster installation.
There are generally two steps to this deployment:
- Installation of KVM on the baremetal system and virtual machine instantiation
- Kubernetes environment installed and setup on the virtual machines
First we start by configuring our
virthost/ inventory to match our working
environment, including DNS or IP address of the baremetal system that we'll
install KVM onto. We also setup our network (KVM network, whether that be a
bridged interface, or a NAT interface), and then define the system topology
we're going to deploy (number of virtual machines to instantiate).
We do this with the
virthost-setup.yml playbook, which performs the virtual
host basic configuration, virtual machine instantiation, and extra virtual disk
creation when configuring persistent storage with GlusterFS.
vms.local.generated inventory file is
created with the IP addresses and hostname of the virtual machines. The
vms.local.generated file can then be used with
Step 0. Install dependent roles
Install role dependencies with
ansible-galaxy install -r requirements.yml
Step 1. Create virtual host inventory
Copy the example
virthost inventory into a new directory.
cp -r inventory/examples/virthost inventory/virthost/
Step 2. Setup virtual host inventory
./inventory/virthost/virthost.inventory to setup a virtual
host (skip to step 3 if you already have an inventory).
Want more VMs? Edit
inventory/virthost/group_vars/virthost.yml and add an
override list via
virtual_machines (template in
roles/ka-init/group_vars/all.yml). You can also define separate vCPU and vRAM
for each of the virtual machines with
default values are setup via
which can also be overridden if you wish different default values. (Current
defaults are 2048MB and 4 vCPU.)
If you're not going to be connecting to the virtual machines from the same network as your source machine, you'll need to make sure you setup the
ssh_proxy_enabled: trueand other related
ssh_proxy_...variables to allow the
kube-install.ymlplaybook to work properly. See next NOTE for more information.
Running on virthost directly
ansible-playbook -i inventory/virthost/ playbooks/virthost-setup.yml
Setting up virthost as a jump host
ansible-playbook -i inventory/virthost/ -e ssh_proxy_enabled=true playbooks/virthost-setup.yml
There are a few extra variables you may wish to set against the virtual host which can be satisfied in the
inventory/virthost/group_vars/virthost.ymlfile of your local inventory configuration in
inventory/virthost/that you just created.
Primarily, this is for overriding the default variables located in the
roles/ka-init/group_vars/all.ymlfile, or overriding the default values associated with the roles.
Some common variables you may wish to override include:
bridge_networking: falsedisable bridge networking setup
images_directory: /home/images/kubelaboverride image directory location
spare_disk_location: /home/images/kubelaboverride spare disk location
The following values are used in the generation of the dynamic inventory file
ssh_proxy_enabled: trueproxy via jump host (remote virthost)
ssh_proxy_user: rootusername to SSH into virthost
ssh_proxy_host: virthosthostname or IP of virthost
ssh_proxy_port: 2222port of the virthost (optional, default 22)
vm_ssh_key_path: /home/lmadsen/.ssh/id_vm_rsapath to local SSH key
Step 3. Install Kubernetes
During the execution of Step 1 a local inventory should have been
generated for you called
inventory/vms.local.generated that contains the
hosts and their IP addresses. You should be able to pass this inventory to the
If you're not running the Ansible playbooks from the virtual host itself, it's possible to connect to the virtual machines via SSH proxy. You can do this by setting up the
ssh_proxy_...variables as noted in Step 2.
Alternatively you can ignore the generated inventory and copy the example
inventory directory from
inventory/examples/vms/ and modify to your hearts
ansible-playbook -i inventory/vms.local.generated playbooks/kube-install.yml
Once you've done that, you should be able to connect to your Kubernetes master
virtual machine and run
kubectl get nodes and see that all your nodes are in
a Ready state. (It may take some time for everything to coalesce and the
nodes to report back to the Kubernetes master node.)
In order to login to the nodes, you may need to
ssh-add ~/.ssh/vmhost/id_vm_rsa. The private key created on the virtual host will be
automatically fetched to your local machine, allowing you to connect to the
nodes when proxying.
You can create a
~/.bashrcalias to SSH into the virtual machines if you're not executing the Ansible playbooks directly from your virtual host (i.e. from your laptop or desktop). To SSH into the nodes via SSH proxy, add the following alias:
alias ssh-virthost='ssh -o ProxyCommand="ssh -W %h:%p root@virthost"'
It's assumed you're logging into the virtual host as the
rootuser and at hostname
virthost. Change as required.
source ~/.bashrc ; ssh-virthost centos@kube-master
Step 4. Checking your installation
Once you're logged into your Kubernetes master node, run the following command to check the state of your cluster.
kubectl get nodes NAME STATUS ROLES AGE VERSION kube-master Ready master 10m v1.8.3 kube-node-1 Ready <none> 9m v1.8.3 kube-node-2 Ready <none> 9m v1.8.3 kube-node-3 Ready <none> 9m v1.8.3
Everything should be marked as ready. If so, you're good to go!
Initially inspired by: