WIP: kubelet/rkt: Support rkt for container runtime. (Splitted into other PRs)

[yifan-gu]

Let's use this PR to track the missing functions for experimental rkt support in k8s:
(Expected version of rkt: 0.5.4)

• Privilege container (though for now, containers by default have CAP_SYS_ADMIN in rkt, see (stage1: rkt pods should not be given CAP_SYS_ADMIN rkt/rkt#576)
• Exec/RunInContainer
• Get Container Logs (Only pod level logs for now)
• Image pull with credentials
• Garbage collection
• Host network
• More status, including PID (Now we have PID, exitcode)
• Unify the container image format, let's assume it's docker images for now. I decided to abort this for now, let's just do docker:// for now until we get a solution for Add RunInContainer/ExecInContainer to container Runtime API. #7208
• Portforwarding

What I am missing? @vmarmol @dchen1107 @jonboulle

Update:

• Get container info

[yifan-gu]

#7202

[dchen1107]

PR description SGTM. Is PR itself ready for review?

[yifan-gu]

@dchen1107 No... I will remove WIP and split the commits once ready :)

[yifan-gu]

/cc @bakins

[smarterclayton]

@pweil- please be aware of this w.r.t. security context so that the changes you are making are appropriately abstractable for Yifan

[pmorie]

We also need to accommodate rocket with pre-start hooks
On Sat, Apr 25, 2015 at 1:53 PM Clayton Coleman notifications@github.com
wrote:

@pweil- https://github.com/pweil- please be aware of this w.r.t.
security context so that the changes you are making are appropriately
abstractable for Yifan

—
Reply to this email directly or view it on GitHub
#7244 (comment)
.

[bcbroussard]

@yifan-gu thanks for the rkt support. I was experimenting with rkt on k8s through a privileged docker container a few days ago. Nice to see it coming in as a first class citizen.

Are you planning to accommodate per namespace trusted keys, maybe through secrets or some other means?

[smarterclayton]

Cc @liggitt re: pull secrets - the delivery method for pull secrets to the kubelet will differ between rocket and docker unless rocket supports .dockercfg. I think we can let supporting pull secrets be a follow on for the rocket plugin.

On Apr 25, 2015, at 3:30 PM, BC Broussard notifications@github.com wrote:

@yifan-gu thanks for the rkt support. I was experimenting with rkt on k8s through a privileged docker container a few days ago. Nice to see it coming in as a first class citizen.

Are you planning to accommodate per namespace trusted keys, maybe through secrets or some other means?

—
Reply to this email directly or view it on GitHub.

[bcbroussard]

Sounds good, thanks @smarterclayton

On Apr 25, 2015, at 12:44 PM, Clayton Coleman notifications@github.com wrote:

Cc @liggitt re: pull secrets - the delivery method for pull secrets to the kubelet will differ between rocket and docker unless rocket supports .dockercfg. I think we can let supporting pull secrets be a follow on for the rocket plugin.

On Apr 25, 2015, at 3:30 PM, BC Broussard notifications@github.com wrote:

@yifan-gu thanks for the rkt support. I was experimenting with rkt on k8s through a privileged docker container a few days ago. Nice to see it coming in as a first class citizen.

Are you planning to accommodate per namespace trusted keys, maybe through secrets or some other means?

―
Reply to this email directly or view it on GitHub.

―
Reply to this email directly or view it on GitHub.

[pweil-]

@smarterclayton - will read through this today. Thanks for the heads up

[vmarmol]

btw can we get the Godeps merged first as a separate PR? GitHub chokes trying to review such a large change and doesn't let us see the important parts of this PR.

[pmorie]

Do you expect to use secrets for credentials for image pull in this PR?

[yifan-gu]

btw can we get the Godeps merged first as a separate PR? GitHub chokes trying to review such a large change and doesn't let us see the important parts of this PR.

Good idea. Will do after one more commit (trying to get pid, exit code in status)

[dchen1107]

+1 on separate PR on Godeps. Thanks!

[yifan-gu]

@pmorie Good question. I haven't plan for using secrets in this PR. I'm not sure what's the plan for secrets for image pulling. Are we going to add a credential provider for secret volumes ??

[pmorie]

@yifan-gu excellent question. We definitely need to support secrets from a
service account, but I don't think you should have to make a volume in a
pod for a secret used for pull.
On Mon, Apr 27, 2015 at 10:02 PM Yifan Gu notifications@github.com wrote:

@pmorie https://github.com/pmorie Good question. I haven't plan for
using secrets in this PR. I'm not sure what's the plan for secrets for
image pulling. Are we going to add a credential provider for secret volumes
??

—
Reply to this email directly or view it on GitHub
#7244 (comment)
.

[smarterclayton]

The plan is no, at this moment - pull secrets are outside of the volumes (because they are managed outside of the containers)

On Apr 27, 2015, at 10:20 PM, Paul Morie notifications@github.com wrote:

@yifan-gu excellent question. We definitely need to support secrets from a
service account, but I don't think you should have to make a volume in a
pod for a secret used for pull.
On Mon, Apr 27, 2015 at 10:02 PM Yifan Gu notifications@github.com wrote:

@pmorie https://github.com/pmorie Good question. I haven't plan for
using secrets in this PR. I'm not sure what's the plan for secrets for
image pulling. Are we going to add a credential provider for secret volumes
??

—
Reply to this email directly or view it on GitHub
#7244 (comment)
.

—
Reply to this email directly or view it on GitHub.

[yifan-gu]

Got it, thanks! @smarterclayton @pmorie

[pmorie]

@yifan-gu @vmarmol Is the plan still to use ContainerRuntime as the interface to docker/rkt from kubelet?

[vmarmol]

@pmorie yes, it may take some time to do completely though.

[yifan-gu]

@vmarmol @dchen1107 I have most the missing functions implemented. Will do a cleanup and rebase.
I will try to group related functions in separate files for easy review.

Note that get container log now mixed up all container's logs within a pod. We are working on that.

Also a few functions' signature needs to sync with runtime.

[yifan-gu]

Splitted into:

#7605
#7599
#7589
#7555
#7553
#7550
#7549
#7545
#7543
#7526

The tests requires host to have rkt and systemd. So I think we need to put it off for now.

Closing!