Ever wanted to manage your CoreOS infrastructure with Puppet? These are the files to make it happen locally with Vagrant.
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
.gitignore
LICENSE
README.md
Vagrantfile
config.rb
coreos-agent-install.sh
puppet-agent-install.sh
puppet-master-install.sh
second-coreos-data
user-data

README.md

Managing Kubernetes and CoreOS with Puppet Demo

Ever wanted to manage your CoreOS infrastructure with Puppet? These are the files to make it happen locally with Vagrant.

Getting Started

Installation

vagrant plugin install vagrant-hosts

Create VMs

First thing's first: bring up some VMs. There are 4 VMs included here: a puppet master, a CentOS puppet agent (for debugging), and 2 CoreOS machines (an extra one for debugging). For this demo, you just need the master and 1 CoreOS agent.

vagrant up puppetmaster coreosagent

Then run puppet on the master to make sure puppet is installed and working correctly.

vagrant ssh puppetmaster
sudo su -
puppet agent -t

Connect CoreOS Agent

The puppet agent is run inside a container on CoreOS. You mount any directories you want to make changes to the container, give it privileges, and then run the puppet agent and it can make changes to the underlying CoreOS system.

vagrant ssh coreosagent
sudo su -
docker run -p 443:443 -p 80:80 --rm --privileged \
-v /etc:/etc \
-v /var:/var \
-v /usr:/usr \
-v /opt/bin:/opt/bin \
--network host puppet/puppet-agent

Sign the cert on the puppet master VM:

puppet cert sign --all

Then run puppet agent again on the CoreOS VM

docker run -p 443:443 -p 80:80 --rm --privileged \
-v /etc:/etc \
-v /var:/var \
-v /usr:/usr \
-v /opt/bin:/opt/bin \
--network host puppet/puppet-agent

And there you have it!

Install and Apply MOTD Module (Optional)

You can verify your setup is working by installing the puppetlabs MOTD module, which writes a message to /etc/motd. This is a good exercise if you're newer to Puppet and modules, but if not I recommend skipping ahead to the Kubernetes steps.

On the puppet master:

puppet module install puppetlabs-motd

Then add the following to /etc/puppetlabs/code/environments/production/manifests/site.pp

node default {
  class { 'motd':
    content => "Hello world!\n",
  }
}

Run puppet on the master to make sure it's working

puppet agent -t
cat /etc/motd

and you should see 'Hello World!' printed.

Then do the same on the CoreOS machine:

docker run -p 443:443 -p 80:80 --rm --privileged \
-v /etc:/etc \
-v /var:/var \
-v /usr:/usr \
-v /opt/bin:/opt/bin \
--network host puppet/puppet-agent

cat /etc/motd

And you should see the same thing!

Deploy a Kubernetes Cluster to CoreOS

I highly recommend going through the entire README of the Puppet Kubernetes module, but here's an abbreviated version:

Setup

Install the module on the master (I chose to manually install):

puppet module install puppetlabs-kubernetes

Generate the module config on the master

docker run --rm -v $(pwd):/mnt -e OS=coreos -e VERSION=1.9.2 \
-e CONTAINER_RUNTIME=docker -e CNI_PROVIDER=flannel -e \
FQDN=coreos-agent \
-e IP="10.20.1.82" \
-e KUBE_IMAGE_TAG="v1.9.3_coreos.0" \
-e BOOTSTRAP_CONTROLLER_IP="10.20.1.82" \
-e ETCD_INITIAL_CLUSTER="etcd-kube-master=http://10.20.1.82:2380" \
-e ETCD_IP="10.20.1.82" \
-e KUBE_API_ADVERTISE_ADDRESS="10.20.1.82" \
-e INSTALL_DASHBOARD=true puppet/kubetool

This will generate a hiera data file kubernetes.yaml in the current working directory. Move that file to where you keep your hieradata

If you're not sure, most likely:

mv kubernetes.yaml /etc/puppetlabs/code/environments/production/data

Install Kubernetes

Then open a file /etc/puppetlabs/code/environments/production/manifests/site.pp with the following:

node 'coreos-agent.my.network.net' {
  class {'kubernetes':
    controller           => true,
    bootstrap_controller => true,
  }
}

On the CoreOS machine run

docker run -p 443:443 -p 80:80 --rm --privileged \
-v /etc:/etc \
-v /var:/var \
-v /usr:/usr \
-v /lib64:/lib64 \
-v /run:/run \
-v /opt/bin:/opt/bin \
--network host puppet/puppet-agent

Networking Issues

I've run into a number of networking issues while setting this up, so here's how to make sure all your ducks are in a row:

Networks

Possibly not relevant anymore

I've found that when I'm on a the VMs append that network to the specified hostname for the VMs. This means that in order for the VMs to talk to each other they actually need to connect to puppet-master.my.network.net instead of just puppet-master. You can find the name of your networkon Ubuntu by TODO.

I've set up the provisioning scrips to read the network name from an environment variable, so if you run into networking issues initially you can set that variable and rebuild your VMs to see if that ameliorates the issue:

hostname myhostname.my.network.net
export HOSTNAME=$HOSTNAME.my.network.net
# And change the hostname in your /etc/hostname file
vi /etc/hostname

/etc/hosts

Make sure that your hosts all know about each other. The vagrant hosts plugin should take care of this for you, but in case something goes awry your /etc/hosts file should look something like this:

On the puppet master:

root@puppet-master:~# cat /etc/hosts
127.0.0.1   localhost
10.20.1.82  coreos-agent    coreosagent
10.20.1.80  puppet-master   puppet-master
127.0.1.1   puppet-master   puppetmaster
127.0.1.1   ubuntu-xenial   ubuntu-xenial

On the coreos agent:

coreos-agent ~ # cat /etc/hosts 
127.0.0.1 localhost
127.0.1.1 coreos-agent coreosagent
10.20.1.82 coreos-agent coreosagent
10.20.1.80 puppet-master puppet-master