Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

k8s: Update deployments + docs to use hyperkube image and kubelet-wrapper #276

Merged
merged 2 commits into from Mar 8, 2016

Conversation

@aaronlevy
Copy link
Contributor

@aaronlevy aaronlevy commented Feb 16, 2016

/cc @robszumski @joshix for doc changes

@aaronlevy aaronlevy force-pushed the aaronlevy:kubelet-wrapper branch 2 times, most recently to eb473ff Feb 16, 2016
@@ -1,6 +1,6 @@
# Deploy Kubernetes Master Node(s)

Boot a single CoreOS machine which will be used as the Kubernetes master node. You must use a CoreOS version 773.1.0+ on the Alpha or Beta channel for the `kubelet` to be present in the image.
Boot a single CoreOS machine which will be used as the Kubernetes master node. You must use a CoreOS version 960.0.0+ for the `kubelet-wrapper` script to be present in the image. If you wish to use an earlier version (e.g. from the 'stable' channel) see [kubelet-wrapper](kubelet-wrapper.md) for more information.

This comment has been minimized.

@robszumski

robszumski Feb 18, 2016
Member

My gut says that this mentions to many "under the hood" details, ie mentioning the wrapper at all. Maybe just say:

"You must use a CoreOS version 960.0.0+ for the correct version of the Kubelet to be in the image."

This comment has been minimized.

@aaronlevy

aaronlevy Feb 18, 2016
Author Contributor

I don't want to say that the kubelet is in the image (because we're going to be removing it). What about:

"You must use a CoreOS version 962.0.0+ for the kubelet-wrapper script to be present in the image. See kubelet-wrapper for more information.

This comment has been minimized.

@robszumski

robszumski Feb 18, 2016
Member

what about a phrase like "Kubelet support" or "required scripts"?

This comment has been minimized.

@robszumski

robszumski Feb 18, 2016
Member

I don't really feel that strongly about this, btw

@@ -0,0 +1,39 @@
# Kubelet Wrapper Script

The kubelet has some unique requirements, so we need to be able to run the kubelet in an unconstrained environment. However, we also want to ship the kubelet as a container image to take advantage of all that has to offer (image discovery, signing/verification, management).

This comment has been minimized.

@robszumski

robszumski Feb 18, 2016
Member

Maybe be a little more straightforward with this?

The kubelet is the orchestrator of containers on each host in the Kubernetes cluster — it starts and stops containers, manipulates iptables rules, and other low-level, essential tasks. In order to accomplish these tasks, the kubelet requires special permissions on the host.

CoreOS recommends running the kubelet using the rkt container engine, because it has the correct set of features to enable these special permissions, while taking advantage of all that container packaging has to offer: image discovery, signing/verification, and simplified management.


The kubelet has some unique requirements, so we need to be able to run the kubelet in an unconstrained environment. However, we also want to ship the kubelet as a container image to take advantage of all that has to offer (image discovery, signing/verification, management).

The kubelet-wrapper is a helper-script shipped with CoreOS versions 960.0.0+. The script allows a deployer to easily run the kubelet as a container on the host system.

This comment has been minimized.

@robszumski

robszumski Feb 18, 2016
Member

CoreOS ships a wrapper script, kubelet-wrapper, which makes it very easy to run the kubelet under rkt. This script accomplishes two things:

  1. Future releases of CoreOS can tweak the system-related parameters of the kubelet, such as mounting in /etc/ssl/certs.
  2. Allows user-specified flags and the desired version of the kubelet to be passed to rkt. This gives each cluster admin control to enable newer API features and tweak settings easily, independent of CoreOS releases.

This script is currently shipping in CoreOS 960.0.0+ and will be included in all channels in the near future.

This comment has been minimized.

@marineam

marineam Feb 18, 2016

962.0.0 is the version

--config=/etc/kubernetes/manifests
```

In the example above we set the `KUBELET_VERSION` and the kubelet-wrapper script takes care of running the correct container image with all of the required options.

This comment has been minimized.

@robszumski

robszumski Feb 18, 2016
Member

In the example above we set the KUBELET_VERSION and the kubelet-wrapper script takes care of running the correct container image with our desired API server address and manifest location.


## Manual deployment

If you wish to use the kubelet-wrapper on a CoreOS version prior to 960.0.0, you can manually place the script on the host.

This comment has been minimized.

@peebs

peebs Feb 18, 2016
Contributor

False alarm, I believe its 962.0.0 now.

@@ -7,7 +7,7 @@ After completing this guide, a deployer will be able to interact with the Kubern

### CoreOS Installation

For all nodes running Kubernetes components (masters & workers), you must use CoreOS version 773.1.0+ on the Alpha or Beta channel for the kubelet to be present in the image.
For all nodes running Kubernetes components (masters & workers), you must use a CoreOS version 960.0.0+ for the `kubelet-wrapper` script to be present in the image. If you wish to use an earlier version (e.g. from the 'stable' channel) see [kubelet-wrapper](kubelet-wrapper.md) for more information.

This comment has been minimized.

@peebs

peebs Feb 18, 2016
Contributor

962.0.0

@@ -1,6 +1,6 @@
# Deploy Kubernetes Master Node(s)

Boot a single CoreOS machine which will be used as the Kubernetes master node. You must use a CoreOS version 773.1.0+ on the Alpha or Beta channel for the `kubelet` to be present in the image.
Boot a single CoreOS machine which will be used as the Kubernetes master node. You must use a CoreOS version 962.0.0+ for the `/usr/lib/coreos/kubelet-wrapper` script to be present in the image. See [kubelet-wrapper](kubelet-wrapper.md) for more information.

See the [CoreOS Documentation](https://coreos.com/os/docs/latest/) for guides on launching nodes on supported platforms.

This comment has been minimized.

@brianredbeard

brianredbeard Feb 20, 2016
Member

It's not a matter of "enhanced security", it's "basic security" - "For security reasons, these secrets should not be stored in cloud-config"

This comment has been minimized.

@aaronlevy

aaronlevy Mar 8, 2016
Author Contributor

ack. Changed below


## Manual deployment

If you wish to use the kubelet-wrapper on a CoreOS version prior to 962.0.0, you can manually place the script on the host.

This comment has been minimized.

@chancez

chancez Feb 22, 2016
Contributor

It might be worth mentioning the minimum required version of coreos that had all the rkt features necessary to do this.

@robszumski
Copy link
Member

@robszumski robszumski commented Feb 25, 2016

We really need to land these docs. What's the current status? Can I help drive this along?

@aaronlevy
Copy link
Contributor Author

@aaronlevy aaronlevy commented Feb 25, 2016

Still working on a stable v1.1.7 image. Taking some time to backport docker v1.10 support. Given recent tests, this should be ready this week.

@robszumski
Copy link
Member

@robszumski robszumski commented Feb 25, 2016

Sounds good. I was mostly talking about the docs side of this, would it be helpful to separate the two?

@aaronlevy
Copy link
Contributor Author

@aaronlevy aaronlevy commented Feb 25, 2016

The docs shouldn't land until there is an image to use. Otherwise it will be broken for anyone using them.

Hopefully the last pieces for an initial release:

Should be in CoreOS-alpha tomorrow: coreos/bugs#1132
Minor change to cAdvisor: google/cadvisor#1125

CoreOS ships a wrapper script, `/usr/lib/coreos/kubelet-wrapper`, which makes it very easy to run the kubelet under rkt. This script accomplishes two things:

1. Future releases of CoreOS can tweak the system-related parameters of the kubelet, such as mounting in /etc/ssl/certs.
1. Allows user-specified flags and the desired version of the kubelet to be passed to rkt. This gives each cluster admin control to enable newer API features and easily tweak settings, independent of CoreOS releases.

This comment has been minimized.

@rutsky

rutsky Mar 2, 2016

Should be "2."

This comment has been minimized.

@aaronlevy

aaronlevy Mar 3, 2016
Author Contributor

When the markdown renders it will be "2." -- this just makes it easier to add new bullet points without renumbering all existing.

This comment has been minimized.

@rutsky

rutsky Mar 3, 2016

Huh, I knew about this behavior, but didn't thought anyone uses it.

One of the features of Markdown is that it's source is very readable: source file is formatted almost in the same way, as generated HTML, so usually numbering is done in the same way as it expected to be in HTML.

But it's your choice. Sorry for the noise!

@james-thimont-bcgdv
Copy link

@james-thimont-bcgdv james-thimont-bcgdv commented Mar 3, 2016

@aaronlevy Do you have an idea when you'll be able to release this?
v1.1.2 of Kubernetes has a bug that prevents EBS volumes being reattached to pods and I really need the fix that comes with v1.1.7

@glerchundi
Copy link

@glerchundi glerchundi commented Mar 7, 2016

/ping

For example:

- Retrieve a copy of the [kubelet-wrapper script](https://github.com/coreos/coreos-overlay/blob/master/app-admin/kubelet-wrapper/files/kubelet-wrapper)
- Place on the host: `/etc/coreos/kubelet-wrapper`

This comment has been minimized.

@joshix

joshix Mar 7, 2016
Contributor

Does this file need mode, e.g., 0744?

This comment has been minimized.

@aaronlevy

aaronlevy Mar 7, 2016
Author Contributor

ack. Added below

@@ -7,7 +7,7 @@ After completing this guide, a deployer will be able to interact with the Kubern

### CoreOS Installation

For all nodes running Kubernetes components (masters & workers), you must use CoreOS version 773.1.0+ on the Alpha or Beta channel for the kubelet to be present in the image.
For all nodes running Kubernetes components (masters & workers), you must use a CoreOS version 962.0.0+ for the `kubelet-wrapper` script to be present in the image. If you wish to use an earlier version (e.g. from the 'stable' channel) see [kubelet-wrapper](kubelet-wrapper.md) for more information.

This comment has been minimized.

@joshix

joshix Mar 7, 2016
Contributor

instead of "masters/workers": "controllers/nodes" (or "controllers/workers") (?)

This comment has been minimized.

@aaronlevy

aaronlevy Mar 7, 2016
Author Contributor

Upstream references "master" and "worker" nodes, and "control plane". I'd like to move toward being in-line with upstream use (even though we had used "controllers" in the past).

See the pretty pics here: http://kubernetes.io/v1.1/docs/admin/high-availability.html#overview

This comment has been minimized.

@joshix

joshix Mar 8, 2016
Contributor

Roger that. I was out of step there.

@aaronlevy aaronlevy force-pushed the aaronlevy:kubelet-wrapper branch Mar 8, 2016
@aaronlevy aaronlevy changed the title [WIP] k8s: Update deployments + docs to use hyperkube image and kubelet-wrapper k8s: Update deployments + docs to use hyperkube image and kubelet-wrapper Mar 8, 2016

- Retrieve a copy of the [kubelet-wrapper script](https://github.com/coreos/coreos-overlay/blob/master/app-admin/kubelet-wrapper/files/kubelet-wrapper)
- Place on the host: `/opt/bin/kubelet-wrapper`
- Make the script exacutable: `chmod +x /opt/bin/kubelet-wrapper`

This comment has been minimized.

@joshix

joshix Mar 8, 2016
Contributor

executable

@joshix
Copy link
Contributor

@joshix joshix commented Mar 8, 2016

The one typo of "executable", then LGTM.

Thanks for the clarity on the naming business.

@aaronlevy aaronlevy force-pushed the aaronlevy:kubelet-wrapper branch to 1b48130 Mar 8, 2016
aaronlevy added a commit that referenced this pull request Mar 8, 2016
k8s: Update deployments + docs to use hyperkube image and kubelet-wrapper
@aaronlevy aaronlevy merged commit 5260998 into coreos:master Mar 8, 2016
1 check passed
1 check passed
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@aaronlevy aaronlevy deleted the aaronlevy:kubelet-wrapper branch Mar 8, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

10 participants