Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
kubeadm generate kubelet serving certs with master(s) CA #1223
Is this a BUG REPORT or FEATURE REQUEST?
kubeadm creates certs under
What you expected to happen?
As a result some apps like the metrics-server cannot collect stats from a secured kubelet because the kubelet has certs signed by a different ca from the K8s master(s)
How to reproduce it (as minimally and precisely as possible)?
Install the metrics-server on run:
$ kubectl -n kube-system logs
Anything else we need to know?
Some more background here
There also steps in there that I followed to fix the issue.
This is what the content of the directory looks like:
@raravena80 thanks for the clarification
Only one side note (might be it can help)
I think this is to pre-generate kubelet's server certs. I tinkered with trying to use the Kubelet flags for TLS server bootstrap and rotate server certs, unfortunately I could not get Kubelet to request a server cert for itself using the bootstrap token. Kubelet ends up falling back to its default behavior for server certs, which is to generate a self-signed one.
To the best of my knowledge, right now the only way around that is to generate Kubelet's server certs out-of-band and place them at a deterministic path and kubelet (configured by kubeadm) will pick it up, and set some kubelet flags accordingly; reference: https://kubernetes.io/docs/reference/command-line-tools-reference/kubelet-tls-bootstrapping/#client-and-serving-certificates
It's the kubelet's identity as a server being presented that needs to be signed by the k8s CA, which comes around to the original question.
There's also some relevant discussion at the end of this thread: #118
I think kubeadm may have to add a CSR approver for server cert requests with a valid bootstrap token, just like it does for client cert requests?
kubelet TLS bootstrapping only generates client certificates for whatever reason:
And kubeadm already does this. Perhaps this is a kubelet feature request?
lets summarize the issue:
the problem with kubeadm here is that we are not passing a couple of flags to the kubelet:
without these flags the kubelet defaults to self-signing it's serving certificate when it first runs, which can be verified with:
with a self-signed certificate instead of certificate signed by the cluster CA (
B) document means to enable this on demand similarly to how @raravena80 did it here: https://stackoverflow.com/questions/53212149/x509-certificate-signed-by-unknown-authority-kubeadm/53218524#53218524
C) as commented by @alexbrand
great summary @neolit123 . Do you know if this will slip onto next cycle or work in progress as we speak ? Asking mainly because of the metrics-server which imo every deployment wants to have it ;)
Any movement on this? I'm running up against it to support autoscaling features within a kubeadm deployed cluster.
Current workaround is to turn off CA checking of the kubelet certificate.