service files for fleet to run a kubernetes cluster
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
README.md
dns-controller.yaml
dns-service.yaml
fleet-hosts.service
kubelet.service
kubernetes-apiserver.service
kubernetes-controller-manager.service
kubernetes-dns.service
kubernetes-env.service
kubernetes-proxy.service
kubernetes-scheduler.service

README.md

Launch a Kubernetes Cluster with Fleet

Purpose

The purpose of this repository was my attempt at demonstrating that it is entirely possible to launch a Kubernetes cluster using solely unit files launched using Fleet across CoreOS machines (and it was!)

Background

There is a boundless bevy of repositories that are used to run Kubernetes, most of which use Ansible, which is a great tool for automating just about anything, including Kubernetes deployments of every shape and size and arrangement.

In looking at fleet, the author of this repository realized what a great scheduler it is and how it's quite possible to use it to run any number of services, including Kubernetes.

There are a few repositories out there that attempt it but it seems they are limited to a single machine.

How this is done

The goal of this repository is to take advantage of the environment that is set up on a stock CoreOS system and a pre-existing etcd cluster.

Using the official, CoreOS Vagrant repo, unit files were developed that use shell magic, Kubernete's etcd entries, as well as CoreOS variables to make this work.

All the info is there and with etcd, it's possible to have everything available to do this cross-node (vs. single).

Description of unit files

  • fleet-hosts.service - Ensures that every host in the cluster has an entry in /etc/hosts on every machine
  • kubernetes-env.service -- Ensures that that environment variables are set up that kubernetes requires. This can be edited for the user's environment
  • kubernetes-api.service -- Starts up the kubernetes API server.
  • kubernetes-controller-manager.service -- Starts up the kubernetes controller. Runs on the same machine as the API server. Uses the entry in etcd for /registry/services/endpoints/default/kubernetes to obtain the IP address of the API server
  • kubernetes-scheduler.service -- Starts the kubernetes scheduler. Runs on the same machine as the API server. Uses the entry in etcd for /registry/services/endpoints/default/kubernetes to obtain the IP address of the API server
  • kubelet.service -- Starts of the kubelet. This is a global service and runs on all machines. Uses the entry in etcd for /registry/services/endpoints/default/kubernetes to obtain the IP address of the API server.
  • kubernetes-proxy.service -- Starts a kubernetes proxy server. Runs on any machine with a kubelet service running.
  • kubernetes-dns.service -- Starts SkyDNS controller and service on the newly minted kubernetes cluster. Uses curl to obtain "template" files and regex the correct API server IP address as well as domain and IP address the service will run on.

To run

The easiest way to run these is to log into one of the coreos servers and check out the code. Clone this repo, submit in this order:

  1. fleetctl start fleet-hosts.service
  2. fleetctl start kube*

That's it! Then have at it:

core@core-01 ~ $ fleetctl start fleet-hosts.service 
Unit fleet-hosts.service 
Triggered global unit fleet-hosts.service start
core@core-01 ~ $ fleetctl start kubernetes-env.service 
Unit kubernetes-env.service 
Triggered global unit kubernetes-env.service start
core@core-01 ~ $ fleetctl start kube*                        
Unit kubelet.service 
Unit kubernetes-apiserver.service inactive
Unit kubernetes-controller-manager.service inactive
Unit kubernetes-proxy.service 
Triggered global unit kubelet.service start
Triggered global unit kubernetes-proxy.service start
Unit kubernetes-apiserver.service launched on 489ce01d.../172.17.8.103
Unit kubernetes-controller-manager.service launched on 489ce01d.../172.17.8.103
core@core-01 ~ $ fleetctl list-units
UNIT                    MACHINE             ACTIVE      SUB
fleet-hosts.service         489ce01d.../172.17.8.103    inactive    dead
fleet-hosts.service         92bbccb6.../172.17.8.101    inactive    dead
fleet-hosts.service         aa3851c0.../172.17.8.102    inactive    dead
kubelet.service             489ce01d.../172.17.8.103    active      running
kubelet.service             92bbccb6.../172.17.8.101    active      running
kubelet.service             aa3851c0.../172.17.8.102    active      running
kubernetes-apiserver.service        489ce01d.../172.17.8.103    active      running
kubernetes-controller-manager.service   489ce01d.../172.17.8.103    active      running
kubernetes-dns.service          489ce01d.../172.17.8.103    activating  start-pre
kubernetes-env.service          489ce01d.../172.17.8.103    inactive    dead
kubernetes-env.service          92bbccb6.../172.17.8.101    inactive    dead
kubernetes-env.service          aa3851c0.../172.17.8.102    inactive    dead
kubernetes-proxy.service        489ce01d.../172.17.8.103    active      running
kubernetes-proxy.service        92bbccb6.../172.17.8.101    active      running
kubernetes-proxy.service        aa3851c0.../172.17.8.102    active      running
kubernetes-scheduler.service        489ce01d.../172.17.8.103    active      running
core@core-01 ~ $ kubectl -s http://172.17.8.103:8080 get pods,rc,services
NAME                    READY        STATUS        RESTARTS   AGE
kube-dns-03r0v          0/3          Pending       0          1d
nginx-198147104-dnpfb   0/1          Pending       0          1d
NAME                    DESIRED      CURRENT       AGE
kube-dns                1            1             2d
NAME                    CLUSTER-IP   EXTERNAL-IP   PORT(S)         AGE
kube-dns                10.1.0.100   <none>        53/UDP,53/TCP   2d
kubernetes              10.1.0.1     <none>        443/TCP         5d
core@core-01 ~ $