K3s server skips needed kubelet arguments.

[dereknola]

Environmental Info:
K3s Version:

K3s Master

Describe the bug:

K3s reports:

Nov 15 17:55:30 ck-c7-0 k3s[12375]: time="2021-11-15T17:55:30-08:00" level=warning msg="Unknown flag --kubelet-arg=max-pods=999 found in config.yaml, skipping\n"

Steps To Reproduce:

• Installed K3s:
  Configure config.yaml with:

kubelet-arg: "max-pods=999"
token=12345

Expected behavior:

K3s server should accept and use those arguments.
Actual behavior:

K3s server skips and ignores kubelet-arg
Additional context / logs:

Backporting

• Needs backporting to older releases: 1.21, e1.21, 1.22

[VestigeJ]

Validated
k3s version v1.22.3+k3s-ef263bd2 (ef263bd)
go version go1.16.8

cat /etc/rancher/k3s/config.yaml

node-label: DEAFBEEF
etcd-s3: true
etcd-s3-bucket: my-backup
notaflag : true 
notaflag : true
kubelet-arg: "max-pods=999"
token: 12345

sudo journalctl -u k3s # shows skipping over invalid config entries.

Nov 19 21:11:27 Justin89 k3s[15846]: time="2021-11-19T21:11:27Z" level=warning msg="Unknown flag --notaflag found in config.yaml, skipping\n"

Shows that the kubelet-arg is correctly applied after skipping over invalid config entries.

Nov 19 21:11:34 Justin89 k3s[15846]: time="2021-11-19T21:11:34Z" level=info msg="Running kubelet --address=0.0.0.0 --anonymous-auth=false --authentication-token-webhook=true --authorization-mode=Webhook --cgroup-driver=cgroupfs --client-ca-file=/var/lib/rancher/k3s/agent/client-ca.crt --cloud-provider=external --cluster-dns=10.9.0.90 --cluster-domain=cluster.local --cni-bin-dir=/var/lib/rancher/k3s/data/b856c1bd996935c10346a4ee685bf01e23389171d1eeb/bin --cni-conf-dir=/var/lib/rancher/k3s/agent/etc/cni/net.d --container-runtime-endpoint=unix:///run/k3s/containerd/containerd.sock --container-runtime=remote --containerd=/run/k3s/containerd/containerd.sock --eviction-hard=imagefs.available<5%,nodefs.available<5% --eviction-minimum-reclaim=imagefs.available=10%,nodefs.available=10% --fail-swap-on=false --healthz-bind-address=127.0.0.1 --hostname-override=justin8 --kubeconfig=/var/lib/rancher/k3s/agent/kubelet.kubeconfig --max-pods=999 --node-labels=DEAFBEEF --pod-manifest-path=/var/lib/rancher/k3s/agent/pod-manifests --read-only-port=0 --resolv-conf=/run/systemd/resolve/resolv.conf --serialize-image-pulls=false --tls-cert-file=/var/lib/rancher/k3s/agent/serving-kubelet.crt --tls-private-key-file=/var/lib/rancher/k3s/agent/serving-kubelet.key"

[rmoreas]

Actualy I used the following notation in config.yaml that works on v1.20.12 and before:

kubelet-arg=max-pods: "999"