New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

k8s: Update deployments + docs to use hyperkube image and kubelet-wrapper #276

Merged
merged 2 commits into from Mar 8, 2016

Conversation

Projects
None yet
10 participants
@aaronlevy
Member

aaronlevy commented Feb 16, 2016

/cc @robszumski @joshix for doc changes

@@ -1,6 +1,6 @@
# Deploy Kubernetes Master Node(s)
Boot a single CoreOS machine which will be used as the Kubernetes master node. You must use a CoreOS version 773.1.0+ on the Alpha or Beta channel for the `kubelet` to be present in the image.
Boot a single CoreOS machine which will be used as the Kubernetes master node. You must use a CoreOS version 960.0.0+ for the `kubelet-wrapper` script to be present in the image. If you wish to use an earlier version (e.g. from the 'stable' channel) see [kubelet-wrapper](kubelet-wrapper.md) for more information.

This comment has been minimized.

@robszumski

robszumski Feb 18, 2016

Member

My gut says that this mentions to many "under the hood" details, ie mentioning the wrapper at all. Maybe just say:

"You must use a CoreOS version 960.0.0+ for the correct version of the Kubelet to be in the image."

This comment has been minimized.

@aaronlevy

aaronlevy Feb 18, 2016

Member

I don't want to say that the kubelet is in the image (because we're going to be removing it). What about:

"You must use a CoreOS version 962.0.0+ for the kubelet-wrapper script to be present in the image. See kubelet-wrapper for more information.

This comment has been minimized.

@robszumski

robszumski Feb 18, 2016

Member

what about a phrase like "Kubelet support" or "required scripts"?

This comment has been minimized.

@robszumski

robszumski Feb 18, 2016

Member

I don't really feel that strongly about this, btw

@@ -0,0 +1,39 @@
# Kubelet Wrapper Script
The kubelet has some unique requirements, so we need to be able to run the kubelet in an unconstrained environment. However, we also want to ship the kubelet as a container image to take advantage of all that has to offer (image discovery, signing/verification, management).

This comment has been minimized.

@robszumski

robszumski Feb 18, 2016

Member

Maybe be a little more straightforward with this?

The kubelet is the orchestrator of containers on each host in the Kubernetes cluster — it starts and stops containers, manipulates iptables rules, and other low-level, essential tasks. In order to accomplish these tasks, the kubelet requires special permissions on the host.

CoreOS recommends running the kubelet using the rkt container engine, because it has the correct set of features to enable these special permissions, while taking advantage of all that container packaging has to offer: image discovery, signing/verification, and simplified management.

The kubelet has some unique requirements, so we need to be able to run the kubelet in an unconstrained environment. However, we also want to ship the kubelet as a container image to take advantage of all that has to offer (image discovery, signing/verification, management).
The kubelet-wrapper is a helper-script shipped with CoreOS versions 960.0.0+. The script allows a deployer to easily run the kubelet as a container on the host system.

This comment has been minimized.

@robszumski

robszumski Feb 18, 2016

Member

CoreOS ships a wrapper script, kubelet-wrapper, which makes it very easy to run the kubelet under rkt. This script accomplishes two things:

  1. Future releases of CoreOS can tweak the system-related parameters of the kubelet, such as mounting in /etc/ssl/certs.
  2. Allows user-specified flags and the desired version of the kubelet to be passed to rkt. This gives each cluster admin control to enable newer API features and tweak settings easily, independent of CoreOS releases.

This script is currently shipping in CoreOS 960.0.0+ and will be included in all channels in the near future.

This comment has been minimized.

@marineam

marineam Feb 18, 2016

962.0.0 is the version

--config=/etc/kubernetes/manifests
```
In the example above we set the `KUBELET_VERSION` and the kubelet-wrapper script takes care of running the correct container image with all of the required options.

This comment has been minimized.

@robszumski

robszumski Feb 18, 2016

Member

In the example above we set the KUBELET_VERSION and the kubelet-wrapper script takes care of running the correct container image with our desired API server address and manifest location.

## Manual deployment
If you wish to use the kubelet-wrapper on a CoreOS version prior to 960.0.0, you can manually place the script on the host.

This comment has been minimized.

@pbx0

pbx0 Feb 18, 2016

Member

False alarm, I believe its 962.0.0 now.

@@ -7,7 +7,7 @@ After completing this guide, a deployer will be able to interact with the Kubern
### CoreOS Installation
For all nodes running Kubernetes components (masters & workers), you must use CoreOS version 773.1.0+ on the Alpha or Beta channel for the kubelet to be present in the image.
For all nodes running Kubernetes components (masters & workers), you must use a CoreOS version 960.0.0+ for the `kubelet-wrapper` script to be present in the image. If you wish to use an earlier version (e.g. from the 'stable' channel) see [kubelet-wrapper](kubelet-wrapper.md) for more information.

This comment has been minimized.

@pbx0

pbx0 Feb 18, 2016

Member

962.0.0

@@ -1,6 +1,6 @@
# Deploy Kubernetes Master Node(s)
Boot a single CoreOS machine which will be used as the Kubernetes master node. You must use a CoreOS version 773.1.0+ on the Alpha or Beta channel for the `kubelet` to be present in the image.
Boot a single CoreOS machine which will be used as the Kubernetes master node. You must use a CoreOS version 962.0.0+ for the `/usr/lib/coreos/kubelet-wrapper` script to be present in the image. See [kubelet-wrapper](kubelet-wrapper.md) for more information.
See the [CoreOS Documentation](https://coreos.com/os/docs/latest/) for guides on launching nodes on supported platforms.

This comment has been minimized.

@brianredbeard

brianredbeard Feb 20, 2016

Member

It's not a matter of "enhanced security", it's "basic security" - "For security reasons, these secrets should not be stored in cloud-config"

This comment has been minimized.

@aaronlevy

aaronlevy Mar 8, 2016

Member

ack. Changed below

## Manual deployment
If you wish to use the kubelet-wrapper on a CoreOS version prior to 962.0.0, you can manually place the script on the host.

This comment has been minimized.

@chancez

chancez Feb 22, 2016

Member

It might be worth mentioning the minimum required version of coreos that had all the rkt features necessary to do this.

@robszumski

This comment has been minimized.

Member

robszumski commented Feb 25, 2016

We really need to land these docs. What's the current status? Can I help drive this along?

@aaronlevy

This comment has been minimized.

Member

aaronlevy commented Feb 25, 2016

Still working on a stable v1.1.7 image. Taking some time to backport docker v1.10 support. Given recent tests, this should be ready this week.

@robszumski

This comment has been minimized.

Member

robszumski commented Feb 25, 2016

Sounds good. I was mostly talking about the docs side of this, would it be helpful to separate the two?

@aaronlevy

This comment has been minimized.

Member

aaronlevy commented Feb 25, 2016

The docs shouldn't land until there is an image to use. Otherwise it will be broken for anyone using them.

Hopefully the last pieces for an initial release:

Should be in CoreOS-alpha tomorrow: coreos/bugs#1132
Minor change to cAdvisor: google/cadvisor#1125

CoreOS ships a wrapper script, `/usr/lib/coreos/kubelet-wrapper`, which makes it very easy to run the kubelet under rkt. This script accomplishes two things:
1. Future releases of CoreOS can tweak the system-related parameters of the kubelet, such as mounting in /etc/ssl/certs.
1. Allows user-specified flags and the desired version of the kubelet to be passed to rkt. This gives each cluster admin control to enable newer API features and easily tweak settings, independent of CoreOS releases.

This comment has been minimized.

@rutsky

rutsky Mar 2, 2016

Should be "2."

This comment has been minimized.

@aaronlevy

aaronlevy Mar 3, 2016

Member

When the markdown renders it will be "2." -- this just makes it easier to add new bullet points without renumbering all existing.

This comment has been minimized.

@rutsky

rutsky Mar 3, 2016

Huh, I knew about this behavior, but didn't thought anyone uses it.

One of the features of Markdown is that it's source is very readable: source file is formatted almost in the same way, as generated HTML, so usually numbering is done in the same way as it expected to be in HTML.

But it's your choice. Sorry for the noise!

@james-thimont-bcgdv

This comment has been minimized.

james-thimont-bcgdv commented Mar 3, 2016

@aaronlevy Do you have an idea when you'll be able to release this?
v1.1.2 of Kubernetes has a bug that prevents EBS volumes being reattached to pods and I really need the fix that comes with v1.1.7

@glerchundi

This comment has been minimized.

glerchundi commented Mar 7, 2016

/ping

For example:
- Retrieve a copy of the [kubelet-wrapper script](https://github.com/coreos/coreos-overlay/blob/master/app-admin/kubelet-wrapper/files/kubelet-wrapper)
- Place on the host: `/etc/coreos/kubelet-wrapper`

This comment has been minimized.

@joshix

joshix Mar 7, 2016

Contributor

Does this file need mode, e.g., 0744?

This comment has been minimized.

@aaronlevy

aaronlevy Mar 7, 2016

Member

ack. Added below

@@ -7,7 +7,7 @@ After completing this guide, a deployer will be able to interact with the Kubern
### CoreOS Installation
For all nodes running Kubernetes components (masters & workers), you must use CoreOS version 773.1.0+ on the Alpha or Beta channel for the kubelet to be present in the image.
For all nodes running Kubernetes components (masters & workers), you must use a CoreOS version 962.0.0+ for the `kubelet-wrapper` script to be present in the image. If you wish to use an earlier version (e.g. from the 'stable' channel) see [kubelet-wrapper](kubelet-wrapper.md) for more information.

This comment has been minimized.

@joshix

joshix Mar 7, 2016

Contributor

instead of "masters/workers": "controllers/nodes" (or "controllers/workers") (?)

This comment has been minimized.

@aaronlevy

aaronlevy Mar 7, 2016

Member

Upstream references "master" and "worker" nodes, and "control plane". I'd like to move toward being in-line with upstream use (even though we had used "controllers" in the past).

See the pretty pics here: http://kubernetes.io/v1.1/docs/admin/high-availability.html#overview

This comment has been minimized.

@joshix

joshix Mar 8, 2016

Contributor

Roger that. I was out of step there.

@aaronlevy aaronlevy changed the title from [WIP] k8s: Update deployments + docs to use hyperkube image and kubelet-wrapper to k8s: Update deployments + docs to use hyperkube image and kubelet-wrapper Mar 8, 2016

@joshix

View changes

Documentation/kubelet-wrapper.md Outdated
- Retrieve a copy of the [kubelet-wrapper script](https://github.com/coreos/coreos-overlay/blob/master/app-admin/kubelet-wrapper/files/kubelet-wrapper)
- Place on the host: `/opt/bin/kubelet-wrapper`
- Make the script exacutable: `chmod +x /opt/bin/kubelet-wrapper`

This comment has been minimized.

@joshix

joshix Mar 8, 2016

Contributor

executable

@joshix

This comment has been minimized.

Contributor

joshix commented Mar 8, 2016

The one typo of "executable", then LGTM.

Thanks for the clarity on the naming business.

aaronlevy added a commit that referenced this pull request Mar 8, 2016

Merge pull request #276 from aaronlevy/kubelet-wrapper
k8s: Update deployments + docs to use hyperkube image and kubelet-wrapper

@aaronlevy aaronlevy merged commit 5260998 into coreos:master Mar 8, 2016

1 check passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

@aaronlevy aaronlevy deleted the aaronlevy:kubelet-wrapper branch Mar 8, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment