First actual commit

Berksfile

@@ -0,0 +1,2 @@
+source 'https://supermarket.chef.io'
+metadata

CHANGELOG.md

@@ -0,0 +1,22 @@
+# Change Log
+All notable changes to this project will be documented in this file.
+This project will adhere to strict [Semantic Versioning](http://semver.org/) starting at v1.0.0.
+
+## [0.6.0] - 2015-12-02
+### Changed
+- Can provision a proxy inside the cluster for HA purposes
+- Can provision multiple masters for API HA
+- Clustered etcd for masters possible at cluster creation
+- Minions can use cluster proxy for API and etcd
+- Can configure where docker stores data
+- Can configure etcd data storage location on masters
+- Can specify a pause container for Kubelet
+
+## [0.3.0] - 2015-11-16
+### Changed
+- First workable release
+- Provisioning of single master
+- Supports creation of docker registry
+- Configures flannel networking
+- Configures docker bridge
+- Configures Kubelets

CONTRIBUTING.md

@@ -0,0 +1,8 @@
+## Contributing
+
+1. Fork the repository on Github
+2. Create a named feature branch (i.e. `add-new-recipe`)
+3. Write your change
+4. Write tests for your change (if applicable)
+5. Run the tests, ensuring they all pass
+6. Submit a Pull Request

Gemfile

@@ -0,0 +1,23 @@
+source 'http://artifactory.dev.bloomberg.com:8080/artifactory/api/gems/ruby-repos/'
+
+gem 'berkshelf'
+gem 'chef-sugar'
+gem 'chef-vault'
+gem 'rake'
+
+group :development do
+  gem 'guard-rspec'
+  gem 'guard-rubocop'
+  gem 'guard-foodcritic'
+end
+
+group :test, :integration do
+  gem 'chefspec'
+  gem 'ci_reporter_rspec'
+  gem 'foodcritic'
+  gem 'kitchen-openstack'
+  gem 'rspec', '~> 3.1'
+  gem 'rubocop'
+  gem 'serverspec'
+  gem 'test-kitchen'
+end

Guardfile

@@ -0,0 +1,22 @@
+guard 'foodcritic', cookbook_paths: '.', cli: '-t ~FC023 -t ~FC005', all_on_start: false do
+  watch(%r{^(?:recipes|libraries|providers|resources)/.+\.rb$})
+  watch('metadata.rb')
+end
+
+# More info at https://github.com/guard/guard#readme
+guard 'rubocop' do
+  watch(%r{^attributes/.+\.rb$})
+  watch(%r{^providers/.+\.rb$})
+  watch(%r{^recipes/.+\.rb$})
+  watch(%r{^resources/.+\.rb$})
+  watch(%r{^libraries/.+\.rb$})
+  watch('metadata.rb')
+end
+
+guard :rspec, cmd: 'rspec', spec_paths: %w(test/spec) do
+  watch(%r{^(recipes|libraries|providers|resources)/(.+)\.rb$}) do |m|
+    "test/spec/#{m[1]}/#{m[2]}_spec.rb"
+  end
+  watch(%r{test/spec/.+\.rb$})
+  watch('test/spec/spec_helper.rb')      { 'spec/unit' }
+end

README.md

@@ -1,2 +1,64 @@
-# kubernetes-cluster-cookbook
-A cookbook for creating highly available, secure kubernetes clusters
+#kubernetes-cluster-cookbook
+=================
+Cookbook for configuring container clusters.
+
+## THIS COOKBOOK IS CURRENTLY A POC AND IS NOT AND MAY NEVER BE READY FOR PRODUCTION USE. THERE IS CURRENTLY NO PROMISE FOR SUPPORT FOR THIS PROJECT
+
+## Supported platforms
+- EL 7.1+
+
+The purpose of this cookbook is to install and configure the proper sevices to create application container clusters. This includes etcd, Kubernetes, Flannel, and Docker- specifically aimed at operating on Enterprise Linux platforms. The best method to using this cookbook is to create a wrapper with specific configurations for your project, adding this cookbook as a dependency.
+
+## How to use
+
+This cookbook assumes you have access to a chef server- however, the cookbook will work fine without it if you override node['kubernetes']['master']['fqdn'] in attributes/minion.rb and node['kubernetes']['etcd']['members'] in attributes/master.rb in insecure mode. Secure mode requires more configuration.
+
+## Insecure mode
+
+- First, configure your masters, The first master converge may fail due to ETCD oddities, I am working for a resolution for this. Just re-converge the masters that fail.
+- Third, configure your minions, making sure they use the proxy server as the master, not the actual masters.
+- Fourth, enjoy!
+
+## Secure mode
+
+Secure mode will configure SSL and TLS connections for all endpoints for etcd and Kubernetes. This is HIGHLY recommended for production-like purposes. This will require large amounts of prep work.
+First, set node['kubernetes']['secure']['enabled'] = 'true' and read below:
+
+### SSL/TLS certs
+
+I highly recommend you use a tool like CFSSL (CloudFlare SSL) to create your certificates, check out https://www.digitalocean.com/community/tutorials/how-to-secure-your-coreos-cluster-with-tls-ssl-and-firewall-rules and start at "Use CFSSL to Generate Self-Signed Certificates"
+
+Masters:
+- node['kubernetes']['etcd']['peer']['ca'] - Certificate authority for managing authentication for master to master connections (etcd sync)
+- node['kubernetes']['etcd']['peer']['cert'] - Certificate the master identifies as for peer connections
+- node['kubernetes']['etcd']['peer']['key'] - Key matching peer certificate
+- node['kubernetes']['etcd']['client']['ca'] - Certificate authority for managing authentication for client to master connections (etcdctl/kubectl/kubelet) 
+- node['kubernetes']['etcd']['client']['cert'] - Certificate the client identifies as for connections
+- node['kubernetes']['etcd']['client']['key'] - Key matching client certificate
+
+Minions:
+- node['kubernetes']['etcd']['client']['cert'] - Certificate the client identifies as for connections
+- node['kubernetes']['etcd']['client']['key'] - Key matching client certificate
+
+NOTE: Peer and Client CA can be the same. This will allow for far simpler setup. However, you may use a different CA to more closely manage security if desired.
+
+## Additional considerations
+
+- Minions get a local HAPROXY setup that does HA type functions between multiple masters if needed
+- Local storage is only supported right now, this is to add speed to containers and to cluster setup
+- ALL attributes used are found with a description in an appropriate attributes file. review and make changes as appropriate.
+
+## Testing
+This project will use [Test Kitchen][1] to execute the [ChefSpec][2] tests
+on a clean virtual machine. By default Test Kitchen will use [Vagrant][3]
+and attempt to start a new virtual machine up from a [default Opscode box][4].
+```shell
+bin/kitchen test default-centos-7.1
+```
+However, no meaningful tests are currently written. This will be fixed.
+
+[1]: https://kitchen.ci
+[2]: https://github.com/sethvargo/chefspec
+[3]: http://vagrantup.com
+[4]: https://github.com/opscode/bento
+

Rakefile

@@ -0,0 +1,16 @@
+require 'ci/reporter/rake/rspec'
+require 'rubocop/rake_task'
+require 'rspec/core/rake_task'
+require 'foodcritic'
+
+RuboCop::RakeTask.new
+RSpec::Core::RakeTask.new(:spec) do |t|
+  t.pattern = 'test/spec/**/*_spec.rb'
+end
+FoodCritic::Rake::LintTask.new do |t|
+  t.options = { fail_tags: ['any'] }
+end
+
+task(:default).clear
+desc 'Run linter and tests'
+task default: %w(rubocop foodcritic ci:setup:rspec spec)

Vagrantfile

@@ -0,0 +1,19 @@
+Vagrant.configure('2') do |config|
+  config.vm.box = 'centos-7.1'
+
+  config.vm.provision :chef_solo do |chef|
+    chef.http_proxy = ENV.fetch('http_proxy', nil)
+    chef.https_proxy = ENV.fetch('https_proxy', nil)
+    chef.no_proxy = ENV.fetch('no_proxy', nil)
+    chef.run_list = %w(webops-base::default)
+    chef.json.merge!(
+      chef_client: {
+        config: {
+          http_proxy: chef.http_proxy,
+          https_proxy: chef.https_proxy,
+          no_proxy: chef.no_proxy
+        }
+      }
+    )
+  end
+end

attributes/default.rb

@@ -0,0 +1,36 @@
+#
+# Cookbook Name:: kubernetes-cluster
+#
+# Copyright (C) 2015 Bloomberg Finance L.P.
+#
+# All rights reserved - Do Not Redistribute
+#
+
+#Enable security
+#This will tell the cookbook to make your cluster more secure
+#This enables SSL certificates, enables authn/authz, and disables insecure ports
+#this will make the cookbook run far more complicated- and will require a lot of
+#pre-work such as manually generating SSL certificates and figuring your own way
+default['kubernetes']['secure']['enabled'] = 'false'
+default['kubernetes']['secure']['directory'] = '/etc/kubernetes/secrets'
+
+#Port to host/access Kubernetes API
+default['kubernetes']['insecure']['apiport'] = '8080'
+default['kubernetes']['secure']['apiport'] = '8443'
+
+#Set port for Kubelet communication 
+default['kubelet']['port'] = '10250'
+
+# Package versions
+default['kubernetes_cluster']['package']['flannel']['version'] = ">= 0.2.0"
+default['kubernetes_cluster']['package']['docker']['name'] = "docker"
+default['kubernetes_cluster']['package']['docker']['version'] = "= 1.8.2"
+default['kubernetes_cluster']['package']['kubernetes_client']['version'] = "= 1.0.3"
+default['kubernetes_cluster']['package']['kubernetes_master']['version'] = "= 1.0.3"
+default['kubernetes_cluster']['package']['kubernetes_node']['version'] = "= 1.0.3"
+default['kubernetes_cluster']['package']['etcd']['version'] = ">= 2.0.0"
+default['kubernetes_cluster']['package']['cockpit']['enabled'] = true
+default['kubernetes_cluster']['package']['cockpit']['version'] = ">= 0.71"
+default['kubernetes_cluster']['package']['docker_registry']['version'] = ">= 0.9.1"
+default['kubernetes_cluster']['package']['bridge_utils']['version'] = ">= 1.5"
+default['kubernetes_cluster']['package']['haproxy']['version'] = ">= 1.5.4"

attributes/master.rb

@@ -0,0 +1,61 @@
+#
+# Cookbook Name:: kubernetes-cluster
+# Attributes:: master
+#
+# Author: Drew Rapenchuk <drapenchuk@bloomberg.net>
+#
+# Copyright 2015, Bloomberg, L.P.
+#
+# All rights reserved - Do Not Redistribute
+#
+#
+
+default['kubernetes']['master'].tap do |master|
+
+  #Set the flannel network size of the cluster, and netlength of each node
+  master['flannel-network'] = '1.80.0.0/16'
+  master['flannel-netlength'] = '24'
+
+  #Set the Kubernetes service network size
+  master['service-network'] = '1.90.0.0/16'
+
+  #Get the Kubernetes master hostname
+  master['fqdn'] = node['fqdn']
+
+  #Set the ssl cert for API when ['kubernetes']['secure']['enabled'] = True
+  master['apicert'] = nil
+
+end
+
+default['kubernetes']['etcd'].tap do |etcd|
+
+  #etcd client and peer communication ports
+  etcd['clientport'] = '2379'
+  etcd['peerport'] = '2380'
+
+  #etcd client name for etcd membership
+  etcd['clientname'] = node['fqdn']
+
+  #etcd directory for data storage
+  etcd['basedir'] = '/var/lib/etcd'
+
+  #etcd token for initial cluster creation
+  etcd['token'] = 'newtoken'
+
+  #List of etcd members
+  #This defaults to chef server search capabilities in etcd.rb unless run in chef solo mode
+  #If you override this, set this to ALL masters eg:
+  #etcd['members'] = ["node1.example.com", "node2.example.com", "node3.example.com"]
+  etcd['members'] = nil
+
+  #Set the ssl ca/cert/key for ETCD peer (master to master) communication when ['kubernetes']['secure']['enabled'] = 'true'
+  etcd['peer']['ca'] = nil
+  etcd['peer']['cert'] = nil
+  etcd['peer']['key'] = nil
+
+  #Set the ssl ca/cert/key for ETCD client (minion to master) communication when ['kubernetes']['secure']['enabled'] = 'true'
+  etcd['client']['ca'] = nil
+  etcd['client']['cert'] = nil
+  etcd['client']['key'] = nil
+
+end

attributes/minion.rb

@@ -0,0 +1,41 @@
+#
+# Cookbook Name:: kubernetes-cluster
+# Attributes:: minion
+#
+# Author: Drew Rapenchuk <drapenchuk@bloomberg.net>
+#
+# Copyright 2015, Bloomberg, L.P.
+#
+# All rights reserved - Do Not Redistribute
+#
+
+default['kubernetes']['minion'].tap do |minion|
+
+  #Set hostname for kubelet
+  minion['kubelet-hostname'] = node['fqdn']
+
+  #Set pause container source in case of network connectivity issues
+  minion['pause-source'] = nil
+
+  #Add custom docker registry- optionally insecure
+  minion['docker-registry'] = nil
+  minion['registry-insecure'] = nil
+
+  #Set docker base directory for local storage- make sure this has plenty of space, optimally its own volume
+  #This directory will be created if it does not exist
+  minion['docker-basedir'] = nil  
+
+end
+
+#Your kubernetes master fqdn OR cluster proxy
+#This defaults to chef server search capabilities in kubelet.rb unless run in chef solo mode
+#If you override this, set this to your master fqdn or cluster proxy fqdn:
+#default['kubernetes']['master']['fqdn'] = "myclusterproxy.example.com"
+#or default['kubernetes']['master']['fqdn'] = "myclustermaster.example.com"
+#ONLY statically set the master fqdn as actual master fqdn if you are not doing an HA setup"
+default['kubernetes']['master']['fqdn'] = nil
+
+#Set the ssl cert/key for ETCD client (minion to master) communication when ['kubernetes']['secure']['enabled'] = 'true'
+default['kubernetes']['etcd']['client']['ca'] = nil
+default['kubernetes']['etcd']['client']['cert'] = nil
+default['kubernetes']['etcd']['client']['key'] = nil

attributes/proxy.rb

@@ -0,0 +1,17 @@
+#
+# Cookbook Name:: kubernetes-cluster
+# Attributes:: proxy
+#
+# Author: Drew Rapenchuk <drapenchuk@bloomberg.net>
+#
+# Copyright 2015, Bloomberg, L.P.
+#
+# All rights reserved - Do Not Redistribute
+#
+
+default['proxy'].tap do |proxy|
+
+  #Set api servers for HA
+  proxy['api']['servers'] = nil
+
+end

attributes/registry.rb

@@ -0,0 +1,24 @@
+# Cookbook Name:: kubernetes-cluster
+# Attributes:: registry
+#
+# Author: Drew Rapenchuk <drapenchuk@bloomberg.net>
+#
+# Copyright 2015, Bloomberg, L.P.
+#
+# All rights reserved - Do Not Redistribute
+#
+
+default['kubernetes']['registry'].tap do |registry|
+
+  #Set port for registry
+  registry['port'] = '5000'
+
+  #Set number of workers for Gunicorn
+  registry['workers'] = '8'
+
+  #Set storage location base for images and registry metadata
+  #Only supports local storage on registry server right now
+  #Full path where 'images' and 'registry' directories will be created
+  registry['storage'] = '/var/docker-registry/'
+
+end