New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kubelet RPM upgrade overwrites /etc/sysconfig/kubelet #70489

Open
mxey opened this Issue Oct 31, 2018 · 6 comments

Comments

Projects
None yet
4 participants
@mxey
Copy link

mxey commented Oct 31, 2018

What happened:

When updating the kubelet RPM from 1.11.3-0.x86_64 to 1.11.4-0.x86_64, /etc/sysconfig/kubelet is overwritten with the package contents again.

What you expected to happen:

/etc/sysconfig/kubelet should be left as-is because it is meant for the operator to override

How to reproduce it (as minimally and precisely as possible):

Install Kubelet on CentOS:

$ docker run -it centos:7

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
EOF

yum install -y kubelet-1.11.3 

Modify /etc/sysconfig/kubelet:

echo "FOO=bar" >>  /etc/sysconfig/kubelet 

[root@7d83157dac5a /]# cat /etc/sysconfig/kubelet 
KUBELET_EXTRA_ARGS=
FOO=bar

Upgrade kubelet:

yum upgrade -y  kubelet-1.11.4 

Check /etc/sysconfig/kubelet:

[root@7d83157dac5a /]# cat /etc/sysconfig/kubelet 
KUBELET_EXTRA_ARGS=

Anything else we need to know?:

Configuration files in RPM packages should be marked as %config(noreplace)

Environment:

  • Kubernetes version (use kubectl version): v1.11.4
  • Cloud provider or hardware configuration: Docker
  • OS (e.g. from /etc/os-release): CentOS Linux 7 (Core)
  • Kernel (e.g. uname -a): Linux 7d83157dac5a 4.9.93-linuxkit-aufs #1 SMP Wed Jun 6 16:55:56 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
  • Install tools: —
  • Others:

/kind bug

@mxey

This comment has been minimized.

Copy link

mxey commented Oct 31, 2018

/sig node

@k8s-ci-robot k8s-ci-robot added sig/node and removed needs-sig labels Oct 31, 2018

@steven-sheehy

This comment has been minimized.

Copy link

steven-sheehy commented Nov 6, 2018

I was about to open a ticket, but I think I'll pile onto this one. I have the exact same issue, but for /etc/default/kubelet on Debian based systems. According to the systemd drop-in:

This is a file that the user can use for overrides of the kubelet args as a last resort. Preferably, the user should use the .NodeRegistration.KubeletExtraArgs object in the configuration files instead. KUBELET_EXTRA_ARGS should be sourced from this file.

But I can't use it for overrides if it can get overwritten anytime the kubelet package is updated.

$ dpkg -L kubelet
/.
/etc
/etc/default
/etc/default/kubelet
/usr
/usr/bin
/usr/bin/kubelet
/lib
/lib/systemd
/lib/systemd/system
/lib/systemd/system/kubelet.service
@steven-sheehy

This comment has been minimized.

Copy link

steven-sheehy commented Nov 30, 2018

Looks like this is due to this file in kubernetes/release repo. @chuckha Can this file be removed?

In systemd Environment can't override the same variable in an EnvironmentFile, the workaround is to create a systemd drop-in with another EnvironmentFile:

$ cat /usr/lib/systemd/system/kubelet.service.d/20-containerd.conf
[Service]
EnvironmentFile=-/usr/etc/kubelet

$ cat /usr/etc/kubelet
KUBELET_EXTRA_ARGS=--container-runtime=remote --container-runtime-endpoint=unix:///var/run/containerd/containerd.sock --runtime-request-timeout=15m
@chuckha

This comment has been minimized.

Copy link
Member

chuckha commented Nov 30, 2018

I find it odd that the kubelet package is doing this https://github.com/kubernetes/release/blob/master/debian/xenial/kubelet/debian/kubelet.install#L3.

I think it makes more sense for that to happen in the kubeadm package, and then, only if the file doesn't exist I think. I have to think about the various scenarios, but I think it's sensible to not have the kubelet writing the /etc/default/kubelet file.

@steven-sheehy

This comment has been minimized.

Copy link

steven-sheehy commented Nov 30, 2018

I don't think it should be in the kubeadm package either. Systemd doesn't require it to be there and its supposed to be a user supplied file. Having kubernetes create/supply it would cause difficulties/conflicts with users including it in their own deb file or when creating it dynamically via their provisioning scripts.

@chuckha

This comment has been minimized.

Copy link
Member

chuckha commented Nov 30, 2018

Thanks for the explanation, I agree and have opened a ticket in kubernetes/release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment