Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

RFC: kube-bootstrap #16077

Closed
wants to merge 14 commits into from
Closed

RFC: kube-bootstrap #16077

wants to merge 14 commits into from

Conversation

justinsb
Copy link
Member

I'm looking for feedback on this idea, because I think it could be a nice way to get from bare-machine to Kubernetes across clouds, not just on AWS.

This proposal adds a program called kube-bootstrap. It is called as part of the instance startup scripts, and parses a JSON file containing configuration. It currently performs just two small tasks that are needed for AWS master bootstrapping - attaching a master volume and setting up a route. The "need" here is that this would allow for running the master in an Auto-Scaling Group on AWS, so that if it crashed the system would auto-recover.

There are a few big-picture ideas here:

  • The bootstrap code is written in Go, not bash & salt. Eventually it could take on more of the functionality that currently lives in bash & salt, in the hopes of making the instance-startup code more reliable & more portable across clouds. (And also easier to move into e.g. kubelet)
  • We store configuration in a JSON object, with a schema backed by Go code. Again this is hopefully more reliable, and should also allow for dynamic reconfiguration in future. Ideally this JSON object would live in the API eventually.
  • We reuse the existing cloud providers, so we get e.g. much more robust attach-volume code than we would otherwise have - and we get it "for free"
  • In the particular case of the master volume, we can treat attaching the volume as a sort of mutex. So we could have a hot-standby machine ready to take over the master volume at all times; it would just spin until it was able to mount the master volume. Or we could have 3 volumes and 3 masters (in an ASG) and get HA master.

All feedback is welcome.

Related to #5472

@k8s-github-robot
Copy link

Labelling this PR as size/XL

@k8s-github-robot k8s-github-robot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Oct 21, 2015
@k8s-bot
Copy link

k8s-bot commented Oct 22, 2015

GCE e2e test build/test passed for commit 9fc95c62e5a89a050b64c4f721fda5a1b15ec93e.

@k8s-github-robot k8s-github-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Oct 22, 2015
@justinsb
Copy link
Member Author

@thockin is this in your wheelhouse? It feels like it's more about kube-up / salt than the stuff (I believe) you typically focus on. And I bug you enough on that stuff as it is...

@thockin
Copy link
Member

thockin commented Oct 23, 2015

I don't have a wheelhouse, or at least it doesn't have walls. :) I'll take
a look, but I'm a little buried with 1.1 prep today :)

On Fri, Oct 23, 2015 at 8:24 AM, Justin Santa Barbara <
notifications@github.com> wrote:

@thockin https://github.com/thockin is this in your wheelhouse? It
feels like it's more about kube-up / salt than the stuff (I believe) you
typically focus on. And I bug you enough on that stuff as it is...

—
Reply to this email directly or view it on GitHub
#16077 (comment)
.

@roberthbailey
Copy link
Contributor

/cc @mikedanese @fgrzadkowski @jszczepkowski

@mikedanese mikedanese assigned mikedanese and unassigned thockin Nov 4, 2015
@justinsb
AWS: Add a strongly typed mountDevice, return mountpoint on detach
7ec6a95 
The strongly-typed mountDevice should fix a mistake I made before (also
fixed in kubernetes#14493), and we want to return the mountpoint so we can use the
updated GCE PD mount logic.
@justinsb
AWS: Allow the mountDevice chosen to be forced
1f0de13 
This is useful for mounting master volumes.
@justinsb
WIP
2a47a7e 
Conflicts:
	pkg/cloudprovider/providers/aws/aws.go
@justinsb
AWS: Separate out download from install steps
421867a 
This doesn't have a big impact on script size because the headers are
stripped by the grep expression.
@justinsb
AWS: Run kube-bootstrap on master
e2d6ff4
@justinsb
Bootstrap: Make bootstrap exit after run
bc1b818 
We make it into more of a daemon long-term but for now it is one-shot.
@justinsb
Bootstrap: misc fixes in cluster/aws
bf5952f
@justinsb
AWS: Workaround for script size bumping up against limit
210bc8c
@justinsb
Bootstrap: Force linking cloud providers
7280e0b
@justinsb
Bootstrap: Mkdir the mount dir first
77fbb95
@justinsb
AWS: Fix log message when EBS device not recognized
aaba84d
@justinsb
Bootstrap: Only exit when bootstrap has succeeded
17a4376
@k8s-github-robot k8s-github-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 6, 2015
@k8s-bot
Copy link

k8s-bot commented Nov 6, 2015

GCE e2e build/test failed for commit 17a4376.

@k8s-bot
Copy link

k8s-bot commented Nov 9, 2015

GCE e2e test build/test passed for commit 052cefe.

@bgrant0607
Copy link
Member

Ref #5754

@bgrant0607 bgrant0607 added the sig/cluster-lifecycle Categorizes an issue or PR as relevant to SIG Cluster Lifecycle. label Nov 11, 2015
@k8s-github-robot k8s-github-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 12, 2015
@ghost ghost added team/control-plane priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. labels Dec 2, 2015
@mikedanese mikedanese assigned ghost and unassigned mikedanese Jan 14, 2016
@mikedanese
Copy link
Member

I think this can be closed

@mikedanese mikedanese closed this Jan 14, 2016
@pires pires mentioned this pull request Jun 22, 2016
Closed
18 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. sig/cluster-lifecycle Categorizes an issue or PR as relevant to SIG Cluster Lifecycle. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
9 participants
@justinsb @k8s-github-robot @k8s-bot @thockin @roberthbailey @bgrant0607 @mikedanese @k8s-teamcity-mesosphere @googlebot