Kubelet startup config flags deprecated - use config file instead

[geerlingguy]

From Kubelet's startup logs via journalctl -f, when running Kubernetes 1.10+:

Flag --cgroup-driver has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Flag --resolv-conf has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Flag --fail-swap-on has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.

See docs: https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/

[geerlingguy]

(Same thing in 1.11... and 1.12 was just released.)

[strican]

Another point is that a user cannot pass in their own config file currently - you get the error can not mix '--config' with arguments [apiserver-advertise-address kubernetes-version pod-network-cidr]. If we used a config, users can override the config and have a baseline config template to copy from and add to.

[stale]

This issue has been marked 'stale' due to lack of recent activity. If there is no further activity, the issue will be closed in another 30 days. Thank you for your contribution!

Please read this blog post to see the reasons why I mark issues as stale.

[stale]

This issue is no longer marked for closure.

[stale]

This issue is no longer marked for closure.

[rdxmb]

As I understand it, we need to create a config-file and set --config <FILE> within the kubelet_args_line. Is this correct? Is there an idea to which path we deploy that config-file?

[rdxmb]

also: I think it would help to create the config file from the templates in the ansible-role instead of lineinfiles like it is in https://github.com/geerlingguy/ansible-role-kubernetes/blob/master/tasks/kubelet-setup.yml

Are there any plans doing that - or doing it in a completely different way?

[rdxmb]

Could we overwrite the line
Environment="KUBELET_CONFIG_ARGS=--config=/var/lib/kubelet/config.yaml"
in /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
to something like
Environment="KUBELET_CONFIG_ARGS=--config=/etc/kubernetes/kubelet-config.yaml"

and generate /etc/kubernetes/kubelet-config.yaml from template?

I think this would be the easiest way to parse variables to this file - as we can just parse them as 1:1 with jinja2-templates.

If you think this is a good way I could try to create a PR for that.

[geerlingguy]

That would be a good idea, yes, but likely it will break existing usage of the role, so it would need to be done in a major version bump or have some backwards compatibility added.

[rdxmb]

Ok, I started to create a PR like discussed above and started testing on my local machine.
However, I've just found, that the kubeletConfiguration can (and should?) be passed with using kubeadm init. See
Does anybody know if this is only for the croup driver? Do we want to use --config with kubeadm init or just with kubectl ?

What do you think?

Edit: Here is the link

https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#configure-cgroup-driver-used-by-kubelet-on-control-plane-node

Or is this just for the cgroup-driver?

... confused ...

Edit again:

Seems like we need to use that with kubeadm init.

Links:

• https://github.com/kubernetes/website/pull/26786/files#diff-7790d405e7f6377d39f6c8a2b74485c6f033400172bda1aade326124ddfba1baR58-R63
• Installing kubeadm should mention about kubeadm config file kubernetes/website#22916
• Kubelet parameters via a config file - implementation not clear kubernetes/website#27188