Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add kubelet cgroup driver property #9879

Merged
merged 1 commit into from
Sep 7, 2020
Merged

Add kubelet cgroup driver property #9879

merged 1 commit into from
Sep 7, 2020

Conversation

bmelbourne
Copy link
Contributor

What type of PR is this?
/kind bug

What this PR does / why we need it:
Allow Docker and Kubelet to set the Cgroup Driver to systemd as recommended in the official Kubernetes docs.

https://kubernetes.io/docs/setup/production-environment/container-runtimes/#cgroup-drivers

Which issue(s) this PR fixes:
When attempting to set the Docker container runtime cgroup driver (native.cgroupdriver) to systemd in the Kops YAML manifest.

...
  docker:
    execOpt:
    - native.cgroupdriver=systemd
...

The following Kubelet error is reported during the Kops Kubernetes bootstrapping process.

Sep 07 10:48:58 ip-172-20-62-0 kubelet[8384]: F0907 10:48:58.919376    8384 server.go:274] 
failed to run Kubelet: misconfiguration: kubelet cgroup driver: "cgroupfs" is different from 
docker cgroup driver: "systemd"

Special notes for your reviewer:
Tested using Kops Version 1.19.0-alpha.3 (git-d8b7310c69).

/etc/sysconfig/kubelet

DAEMON_ARGS="--anonymous-auth=false --authentication-token-webhook=true 
--authorization-mode=Webhook --cgroup-driver=systemd --cgroup-root=/ 
--client-ca-file=/srv/kubernetes/ca.crt --cloud-provider=aws --cluster-dns=
100.64.0.10 --cluster-domain=cluster.local --enable-debugging-handlers=true 
--eviction-hard=memory.available<100Mi,nodefs.available<10%,nodefs.inodesFree<5%,imagefs.
available<10%,imagefs.inodesFree<5% --hostname-override=ip-172-20-50-252.
eu-west-2.compute.internal --kubeconfig=/var/lib/kubelet/kubeconfig --network-plugin=cni 
--non-masquerade-cidr=100.64.0.0/10 --pod-infra-container-image=k8s.gcr.io/pause:3.2 
--pod-manifest-path=/etc/kubernetes/manifests --register-schedulable=true 
--register-with-taints=node-role.kubernetes.io/master=:NoSchedule --resolv-conf=
/run/systemd/resolve/resolv.conf --v=2 --volume-plugin-dir=/usr/libexec/kubernetes/
kubelet-plugins/volume/exec/ --cni-bin-dir=/opt/cni/bin/ --cni-conf-dir=/etc/cni/net.d/"
HOME="/root"

@k8s-ci-robot k8s-ci-robot added kind/bug Categorizes issue or PR as related to a bug. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Sep 7, 2020
@k8s-ci-robot
Copy link
Contributor

Welcome @bmelbourne!

It looks like this is your first PR to kubernetes/kops 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes/kops has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

@k8s-ci-robot k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Sep 7, 2020
@k8s-ci-robot
Copy link
Contributor

Hi @bmelbourne. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Sep 7, 2020
@hakman
Copy link
Member

hakman commented Sep 7, 2020

Thanks for adding this. Was quite curious to try it myself.
Please run make crds and commit the changes.
/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Sep 7, 2020
@hakman hakman removed the kind/bug Categorizes issue or PR as related to a bug. label Sep 7, 2020
Copy link
Member

@hakman hakman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think "explicit" is misspelled. Please fix and also squash all commits.

k8s/crds/kops.k8s.io_clusters.yaml Outdated Show resolved Hide resolved
k8s/crds/kops.k8s.io_clusters.yaml Outdated Show resolved Hide resolved
k8s/crds/kops.k8s.io_instancegroups.yaml Outdated Show resolved Hide resolved
pkg/apis/kops/componentconfig.go Outdated Show resolved Hide resolved
pkg/apis/kops/v1alpha2/componentconfig.go Outdated Show resolved Hide resolved
@bmelbourne
Copy link
Contributor Author

bmelbourne commented Sep 7, 2020

I think "explicit" is misspelled. Please fix and also squash all commits.

I'm following most of the steps outlined here...
https://github.com/kubernetes/community/blob/master/contributors/guide/pull-requests.md#run-local-verifications

But as I'm running on a Windows 10 machine and hence no Docker available, it's leaving me dependent on the outcome of the PR tests for feedback.

...
hack/verify-spelling.sh
/c/Google_Drive/GitHub/kops/hack/verify-shellcheck.sh
Using shellcheck 0.7.1 docker image.
Failed to create shellcheck container with output: 

docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?.
See 'docker run --help'.
Makefile:452: recipe for target 'verify-shellcheck' failed
make: *** [verify-shellcheck] Error 1

Also, I'll squash the commits once all of the tests are passing.

@hakman
Copy link
Member

hakman commented Sep 7, 2020

No worries, sometimes it's easier to just wait for automated tests to tell you what's left.
Looks good now. Will wait for the squash and then merge it.

@bmelbourne
Copy link
Contributor Author

/assign @zetaab

@hakman
Copy link
Member

hakman commented Sep 7, 2020

Thanks @bmelbourne .
/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Sep 7, 2020
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bmelbourne, hakman

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 7, 2020
@hakman
Copy link
Member

hakman commented Sep 7, 2020

/retest

@olemarkus
Copy link
Member

It would have been nice with some validation logic here

@hakman
Copy link
Member

hakman commented Sep 7, 2020

It would have been nice with some validation logic here

Good idea, can be done in a future PR.

@k8s-ci-robot k8s-ci-robot merged commit bb63f29 into kubernetes:master Sep 7, 2020
@k8s-ci-robot k8s-ci-robot added this to the v1.19 milestone Sep 7, 2020
k8s-ci-robot added a commit that referenced this pull request Sep 7, 2020
…pstream-release-1.18

Automated cherry pick of #9879: Add kubelet cgroup driver property
@bmelbourne bmelbourne deleted the add-kubelet-cgroup-driver branch September 7, 2020 19:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/api cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants