New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rkt: Support alternate stage1's via annotation #25177

Merged
merged 2 commits into from May 25, 2016

Conversation

Projects
None yet
7 participants
@euank
Member

euank commented May 4, 2016

This provides a basic implementation for setting a stage1 on a per-pod
basis via an annotation.

This provides a basic implementation for setting a stage1 on a per-pod
basis via an annotation. See discussion here for how this approach was arrived at: #23944 (comment)

It's possible this feature should be gated behind additional knobs, such
as a kubelet flag to filter allowed stage1s, or a check akin to what
priviliged gets in the apiserver.
Currently, it checks AllowPrivileged, as a means to let people disable
this feature, though overloading it as stage1 and privileged isn't
ideal.

Fixes #23944

Testing done (note, unfortunately done with some additional ./cluster changes merged in):

$ cat examples/stage1-fly/fly-me-to-the-moon.yaml
apiVersion: v1
kind: Pod
metadata:
  labels:
    name: exit
  name: exit-fast
  annotations: {"rkt.alpha.kubernetes.io/stage1-name-override": "coreos.com/rkt/stage1-fly:1.3.0"}
spec:
  restartPolicy: Never
  containers:
    - name: exit
      image: busybox
      command: ["sh", "-c", "ps aux"]
$ kubectl create -f examples/stage1-fly
$ ssh core@minion systemctl status -l --no-pager k8s_2f169b2e-c32a-49e9-a5fb-29ae1f6b4783.service
...
failed
...
May 04 23:33:03 minion rkt[2525]: stage0: error writing /etc/rkt-resolv.conf: open /var/lib/rkt/pods/run/2f169b2e-c32a-49e9-a5fb-29ae1f6b4783/stage1/rootfs/etc/rkt-resolv.conf: no such file or directory
...
# Restart kubelet with allow-privileged=false
$ kubectl create -f examples/stage1-fly
$ kubectl describe exit-fast
...
  1m        19s     5   {kubelet euank-e2e-test-minion-dv3u}    spec.containers{exit}   Warning     Failed      Failed to create rkt container with error: cannot make "exit-fast_default(17050ce9-1252-11e6-a52a-42010af00002)": running a custom stage1 requires a privileged security context
....

Note as well that the "success" here is rkt spitting out an error message which indicates that the right stage1 was being used at least.

cc @yifan-gu @aaronlevy

@googlebot googlebot added the cla: yes label May 4, 2016

Show outdated Hide outdated pkg/kubelet/rkt/rkt.go
@yifan-gu

This comment has been minimized.

Show comment
Hide comment
@yifan-gu

yifan-gu May 5, 2016

Member

LGTM, cc @bgrant0607 @philips @kubernetes/sig-node @pb0

Member

yifan-gu commented May 5, 2016

LGTM, cc @bgrant0607 @philips @kubernetes/sig-node @pb0

@yifan-gu

This comment has been minimized.

Show comment
Hide comment
@yifan-gu

yifan-gu May 5, 2016

Member

Applying LGTM, feel free to remove if you guys have any concerns or questions :)
cc @bgrant0607 @philips @kubernetes/sig-node @pb0

Member

yifan-gu commented May 5, 2016

Applying LGTM, feel free to remove if you guys have any concerns or questions :)
cc @bgrant0607 @philips @kubernetes/sig-node @pb0

@bprashanth

This comment has been minimized.

Show comment
Hide comment
@bprashanth

bprashanth May 16, 2016

Member

your failure is #25375, but you need a rebase anyway

Member

bprashanth commented May 16, 2016

your failure is #25375, but you need a rebase anyway

@yifan-gu

This comment has been minimized.

Show comment
Hide comment
@yifan-gu

yifan-gu May 16, 2016

Member

pkg/kubelet/rkt/rkt.go:986: r.runCommand undefined (type *Runtime has no field or method runCommand, but does have RunCommand)

Member

yifan-gu commented May 16, 2016

pkg/kubelet/rkt/rkt.go:986: r.runCommand undefined (type *Runtime has no field or method runCommand, but does have RunCommand)

@yifan-gu yifan-gu added the lgtm label May 20, 2016

@yifan-gu

This comment has been minimized.

Show comment
Hide comment
@yifan-gu

yifan-gu May 20, 2016

Member

@k8s-bot ok to test.

Member

yifan-gu commented May 20, 2016

@k8s-bot ok to test.

euank added some commits May 4, 2016

rkt: Support alternate stage1's via annotation
This provides a basic implementation for setting a stage1 on a per-pod
basis via an annotation.

It's possible this feature should be gated behind additional knobs, such
as a kubelet flag to filter allowed stage1s, or a check akin to what
priviliged gets in the apiserver.
Currently, it checks `AllowPrivileged`, as a means to let people disable
this feature, though overloading it as stage1 and privileged isn't
ideal.
@euank

This comment has been minimized.

Show comment
Hide comment
@euank

euank May 23, 2016

Member

This'll rebase conflict again once #25902 gets in.

Member

euank commented May 23, 2016

This'll rebase conflict again once #25902 gets in.

@k8s-bot

This comment has been minimized.

Show comment
Hide comment
@k8s-bot

k8s-bot commented May 25, 2016

GCE e2e build/test passed for commit 136da15.

@k8s-merge-robot

This comment has been minimized.

Show comment
Hide comment
@k8s-merge-robot

k8s-merge-robot May 25, 2016

Contributor

Automatic merge from submit-queue

Contributor

k8s-merge-robot commented May 25, 2016

Automatic merge from submit-queue

@k8s-merge-robot k8s-merge-robot merged commit 025b017 into kubernetes:master May 25, 2016

4 of 5 checks passed

Submit Queue Github CI tests are not green.
Details
Jenkins GCE Node e2e Build finished.
Details
Jenkins GCE e2e 305 tests run, 122 skipped, 0 failed.
Details
Jenkins unit/integration 6414 tests run, 26 skipped, 0 failed.
Details
cla/google All necessary CLAs are signed

@euank euank deleted the euank:rkt-alternate-stage1 branch May 25, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment