The goal for this project is to provide a fully functional BOSH CPI. It will eventually allow to deploy any BOSH release to any Kubernetes cluster.
Haskell Shell HTML
Clone or download
loewenstein Remove extra-deps from stack.yml
Dependencies are meanwhile available in stackage snapshot.
Latest commit 150139f Jan 25, 2018
Permalink
Failed to load latest commit information.
.final_builds New final release v6 Sep 26, 2017
ci Bump ci-stack to stack 1.6.3 Jan 24, 2018
config Add keys and blob signatures Jan 25, 2018
dependencies Change location of dependencies.source.tgz Aug 1, 2017
deployment
jobs/kubernetes_cpi
keys Add keys and blob signatures Jan 25, 2018
packages
releases/bosh-kubernetes-cpi New final release v6 Sep 26, 2017
src/bosh-kubernetes-cpi Remove extra-deps from stack.yml Jan 25, 2018
stemcell Adapt to stemcell scripts Sep 1, 2017
.gitignore Test drive re-implementation of attach_disk Sep 22, 2017
.gitmodules
.stylish-haskell.yaml WiP PoC for kubernetes bosh cpi Feb 9, 2017
CONTRIBUTING.md Add contributing.md Oct 13, 2017
LICENSE Add gmp to list of subcomponents Jul 12, 2017
NOTICE Add NOTICE Mar 8, 2017
README.md Update README for Kubernetes CPI v6 Sep 26, 2017

README.md

BOSH Kubernetes CPI

NOTE: This documentation is currently valid for the upcoming release v6.

Description

This project aims to provide a fully functional BOSH CPI for Kubernetes.

There are limitations considering production readiness. However, you can in general use Kubernetes as yet another IaaS in the BOSH ecosystem.

You can find details on what is currently possible and what are known limitations below.

Prerequisites

Installation & Configuration

Get a cluster

Below are the installation steps for a Minikube and a GCE cluster. The same or similar steps should work on any other cluster.

Minikube

$ minikube start --cpus 3 --memory 4096 --disk-size 80g --vm-driver virtualbox
$ minikube ssh -- sudo ip link set docker0 promisc on
  • Prepare cluster
$ kubectl create -f deployment/minikube/assets/
  • Create a BOSH environment
$ ./deployment/minikube/scripts/create-env.sh
$ minikube ssh -- chmod 777 /mnt/sda1/hostpath-provisioner/*

GCE

  • Get an account
  • Install gcloud cli
  • Create a cluster
  • Prepare cluster
$ kubectl create -f deployment/gce/assets/
  • Wait for the loadbalancer to get an external IP
  • Create a BOSH environment
$ CLUSTER_NAME=<cluster name> ./deployment/gce/scripts/create-env.sh

Use the BOSH environment

  • Deploy e.g. zookeeper
$ cd ~/workspace
$ git clone https://github.com/cppforlife/zookeeper-release.git
$ cd ./deployments/minikube/bosh-env
$ bosh update-cloud-config ../bosh-deployment/kubernetes/cloud-config.yml
$ bosh upload-stemcell https://s3.eu-central-1.amazonaws.com/bosh-kubernetes-cpi-stemcells/bosh-stemcell-3445.11-kubernetes-ubuntu-trusty-go_agent.tgz

$ bosh -d zookeeper deploy ~/workspace/zookeeper-release/manifests/zookeeper.yml
$ bosh -d zookeeper deploy ~/workspace/zookeeper-release/manifests/zookeeper.yml
  • Verify Zookeeper deployment
$ bosh -d zookeeper run-errand smoke_tests

Mapping BOSH to Kubernetes

Virtual Machine

A Pod with a single container is used to represent a BOSH virtual machine.

Network

Kubernetes CPI currently only supports dynamic networks.

Persistent Disks

PersistentVolumeClaims are used to represent BOSH persistent disks.

Limitations

  • Only dynamic networks are supported, hence releases that require static IPs are out for now.
  • attach_disk is implemented by deleting the Pod and creating a new one with the required persistent disk attached.
    • there is currently an issue with BOSH DNS and the recreation of the Pod through attach_disk. #2
  • bosh agent requires to run in privileged containers
  • communication with Kubernetes is always insecure (i.e. no SSL verification)
  • legacy authorization
  • limited experimentation with nested containers. Installing Garden (and therefore Diego) is probably not possible at the moment
  • stemcell is currently taken from https://hub.docker.com/r/loewenstein/bosh-stemcell-kubernetes-ubuntu-trusty-go_agent/

Copyright and license

Copyright (c) 2017 SAP SE

Except as provided below, this software is licensed under the Apache License, Version 2.0 (the "License"); you may not use this software except in compliance with the License.You may obtain a copy of the License at:

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.