external etcd support #68

jnummelin · 2018-03-13T13:27:03Z

Support for external etcd configuration, needs top level etcd config in the cluster yaml.

TODOs:

certs for etcd
make sure it works with cri-o too
take in official 1.10 kubeadm when it's released. should come out next week or so

fixes #46

jnummelin · 2018-03-14T13:52:30Z

Certificate setup seems to have some issue on kubeadm side: kubernetes/kubeadm#730

add some specs for master config generator add etcd cert support use certs for external etcd

jnummelin · 2018-03-15T09:38:53Z

Should we wait for official 1.10 kubeadm or merge this with the "hack" to get 1.10.beta kubeadm in and have separate issue to fix that when official release comes out?

jakolehm · 2018-03-15T10:38:47Z

Gemfile.lock

@@ -0,0 +1,130 @@
+PATH


Probably should not be part of this PR?

yeah, I'll take it out

jakolehm · 2018-03-15T10:39:26Z

examples/vagrant/cluster-external-etcd.yml

+    user: vagrant
+    role: master
+    ssh_key_path: ~/.vagrant.d/insecure_private_key
+    container_runtime: docker


No need to define container_runtime, it defaults to docker.

but does not hurt either :)

jakolehm · 2018-03-15T10:42:04Z

lib/kupo/phases/configure_master.rb

-        options << "--apiserver-advertise-address #{@master.address}"
-      end
+      cfg = generate_config
+      tmp_file = File.join('/tmp', 'kubeadm.cfg.' + SecureRandom.hex(16))


This should be removed after init?

Yes, I'll make it happen

jakolehm · 2018-03-15T10:43:31Z

lib/kupo/phases/configure_master.rb

+      @ssh.upload(StringIO.new(cfg.to_yaml), tmp_file)
+
+      # Copy etcd certs over if needed
+      exec_script('configure-etcd-certs.sh', {


What is the reasoning behind this vs using scp to transfer files?

We can't scp into /etc/.... This makes it happen without scp+exec('sudo mv ...')

jakolehm · 2018-03-15T10:44:37Z

lib/kupo/scripts/configure-kube.sh

+# See: https://github.com/kubernetes/kubernetes/pull/59057
+# FIXME Remove when we're using official 1.10 kubeadm
+curl -o /usr/bin/kubeadm https://storage.googleapis.com/kubernetes-release/release/v1.10.0-beta.3/bin/linux/amd64/kubeadm


amd64 is hardcoded.. should probably be <%= arch.name %>.

And if we download binary we don't need kubeadm from repos?

I think we need to figure out the arch in the script itself as it is running on the actual host. Makes it more robust IMO.

We need also some kubeadm version override to hande this kind of case where we run different versions of kubeadm and kube itself.

jakolehm · 2018-03-15T10:45:36Z

spec/kupo/phases/configure_master_spec.rb

@@ -0,0 +1,116 @@
+require "kupo/config"
+require "kupo/phases/configure_master"
+


leftovers from earlier style of cert handling, removed now

jakolehm · 2018-03-15T10:45:49Z

spec/kupo/phases/configure_master_spec.rb

+
+
+  describe '#call' do
+


Something missing here?

jnummelin · 2018-03-15T13:41:37Z

fixed all the commented issues, PTAL @jakolehm

jakolehm · 2018-03-15T14:34:22Z

lib/kupo/scripts/configure-kube.sh

-curl -o /usr/bin/kubeadm https://storage.googleapis.com/kubernetes-release/release/v1.10.0-beta.3/bin/linux/amd64/kubeadm
+# Get kubeadm binary directly
+arch=`uname -m`
+case "$arch" in


I meant that arch should come as a variable from phase. We already collect host.cpu_arch information.

jakolehm

Docs missing (README).

jakolehm · 2018-03-16T06:59:23Z

README.md

@@ -50,6 +50,25 @@ You can view full sample of cluster.yml [here](./cluster.example.yml).
 - `pod_network_cidr` - IP address range for the pod network. (default "10.32.0.0/12")
 - `trusted_subnets` - array of trusted subnets where overlay network can be used without IPSEC.

+## Using external Etcd
+
+Kupo can spin up Kubernetes using an externally managed Etcd. In this case you need to define the external etcd details in your `cluster.yml` file:


Etcd -> etcd (lowercase)

jnummelin force-pushed the feature/external-etcd branch from 4724368 to e3caf38 Compare March 15, 2018 09:22

initial external etcd support

c37eee6

add some specs for master config generator add etcd cert support use certs for external etcd

jnummelin force-pushed the feature/external-etcd branch from e3caf38 to c37eee6 Compare March 15, 2018 09:30

jnummelin changed the title ~~[WIP] external etcd support~~ external etcd support Mar 15, 2018

jnummelin requested review from jakolehm and SpComb March 15, 2018 10:07

add note why we need the kubeadm beta binary

8c3a68d

jakolehm reviewed Mar 15, 2018

View reviewed changes

jnummelin added 3 commits March 15, 2018 13:21

make config struct to use schema validation in constructor

9512e98

cleanup master config spec

f32d9fe

download kubeadm directly, with possibility to override version

d1b859b

jakolehm requested changes Mar 15, 2018

View reviewed changes

use collected cpu arch info for downloading kubeadm

ba6ac5b

jakolehm added the enhancement New feature or request label Mar 16, 2018

jakolehm added this to the 0.3 milestone Mar 16, 2018

jakolehm requested changes Mar 16, 2018

View reviewed changes

jnummelin added 3 commits March 16, 2018 08:48

Merge branch 'master' into feature/external-etcd

9d0eb1f

document external etcd details

467b677

add etcd endpoints info

d74486d

jakolehm reviewed Mar 16, 2018

View reviewed changes

lowercase etcd

e8ba9af

jakolehm approved these changes Mar 16, 2018

View reviewed changes

jnummelin merged commit 5be93dd into master Mar 16, 2018

jnummelin deleted the feature/external-etcd branch March 16, 2018 07:35

jnummelin mentioned this pull request Mar 16, 2018

Use static binary for kubeadm #15

Closed

This was referenced Mar 16, 2018

Hardcoded kube version in config #95

Closed

Missing kubelet dropin #96

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

external etcd support #68

external etcd support #68

jnummelin commented Mar 13, 2018 •

edited

jnummelin commented Mar 14, 2018

jnummelin commented Mar 15, 2018

jakolehm Mar 15, 2018

jnummelin Mar 15, 2018

jakolehm Mar 15, 2018

jnummelin Mar 15, 2018

jakolehm Mar 15, 2018

jnummelin Mar 15, 2018

jnummelin Mar 15, 2018

jakolehm Mar 15, 2018

jnummelin Mar 15, 2018

jakolehm Mar 15, 2018

jakolehm Mar 15, 2018

jnummelin Mar 15, 2018

jnummelin Mar 15, 2018

jakolehm Mar 15, 2018

jnummelin Mar 15, 2018

jakolehm Mar 15, 2018

jnummelin commented Mar 15, 2018

jakolehm Mar 15, 2018

jnummelin Mar 16, 2018

jakolehm left a comment

jakolehm Mar 16, 2018

jnummelin Mar 16, 2018

		@@ -0,0 +1,116 @@
		require "kupo/config"
		require "kupo/phases/configure_master"

external etcd support #68

external etcd support #68

Conversation

jnummelin commented Mar 13, 2018 • edited

jnummelin commented Mar 14, 2018

jnummelin commented Mar 15, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jnummelin commented Mar 15, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jakolehm left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jnummelin commented Mar 13, 2018 •

edited