[design doc] Certificate Rotation

[briandowns]

Problem Statement

K3s has no facilities to regenerate certificates once generated. Certificates are generated when the node bootstraps and those certificates are used through the course of the node's lifetime. The features outlined before are based on an analysis of RKE's features and capabilities for certificate rotation.

Proposal

We want to add an additional sub-command, "cert", to k3s and rke2. This sub-command will have additional commands underneath it. Any "write" operation will perform a backup of the resource prior to performing the requested action. These backups will be placed in a directory named with the UNIX timestamp in the tls directory.

Items marked TBD are out of scope for this round of development effort.

1. rotate:
   1. Rotate should be able to rotate certificates for a given component or all components
      1. Components:
         1. kube-apiserver
         2. kube-scheduler
         3. kube-controller-manager
         4. kubelet
         5. kube-proxy
         6. etcd
2. extend-expiry: TBD
   1. Extend the expiry should only extend the expiration date of the certificates of associated components without changing keys.
3. info: TBD
   1. Print out to STDOUT general info for a given certificate or for all certificates. This should include ciphers, expiration, etc.
4. generate-csr: TBD
   1. Generate certificate signing requests

Examples

k3s cert rotate <component> 
k3s cert rotate --all
k3s cert generate-csr
k3s cert extend-expiry kube-scheduler
k3s cert extend-expiry --all

Additional Considerations

The DynamicListener component manages it's own certificate / key pair and this certificate should be rotated with the rest. The DynamicListener receives a "config" value for it's configuration. To avoid changing the public API, we can add a field to the config struct of type func() bool. K3s will set this function after determining whether or not certificates need to be regenerated by looking for a file on disk set by the k3s cert rotate command.

The Rancher integration would be done via the System-Agent receiving a "certificate-rotation" plan populated with the necessary commands to perform the rotation.

Alternative Approaches

Related Issues

• [epic] - Improved and Manual Cert Rotation #2636

LOE Estimate

• Dynamic Listener: 3 days
• K3s sub-command(s): 2 weeks

[galal-hussein]

Adding more information to the issue, this will include some information about how we implement and generate certificates in k3s/rke2:

Server Node

Generating certs in server nodes is divided into 4 function calls:

genClientCerts()

This function generates the following certs:

• client-ca.crt / client-ca.key (Signing Cert)
• client-admin.crt / client-admin.key
• client-controller.crt / client-controller.key (kube-controller-manager)
• client-scheduler.crt / client-scheduler.key (kube-scheduler)
• client-kube-apiserver.crt / client-kube-apiserver.key (kube-api-server)
• client-rke2-controller.crt / client-rke2-controller.key (system:rke2-controller ??)
• client-rke2-cloud-controller.crt / client-rke2-cloud-controller.crt (rke2 cloud controller manager)

Note: k3s/rke2 generates a kubeconfig for each of these components that uses the certs except for the client-rke2-controller cert and key.

genServerCerts()

This function generates the following certs:

• server-ca.crt/ server-ca.key (Signing Cert)
• serving-kube-apiserver.crt / serving-kube-apiserver.key (kubeapi server cert)
• serving-kubelet.key (This is the key that will be used by server handler to generate kubelet serving cert for the agent nodes and embedded agent in the server)

genRequestHeaderCerts()

• request-header-ca.crt/request-header-ca.key (signing cert)
• client-auth-proxy.crt/client-auth-proxy.key (client auth proxy cert for aggregation layer)

genETCDCerts()

• server-ca.crt/server-ca.key (Signing Cert)
• server-client.crt/server-client.key
• client.crt/client.key
• peer-ca.crt/peer-ca.key (Signing Cert)
• peer-server-client.crt/peer-server-client.key

Agent Node

Agent gets its own certificates for kube proxy and kubelet in agent/config/config.go in get() function, basically it calls server with the --token and --server url to acquire the following certs:

serving-kubelet.crt / serving-kubelet.key (to be used with kubelet’s api server)

A request is initiated from the agent to the server on URI /v1-rke2/serving-kubelet.crt. The server will generate this using the serving kubelet key it generated earlier (see above in genServerCerts() section)

client-kubelet.crt/client-kubelet.key (to be used in kubelet’s kubeconfig file)

A request is initiated from the agent to the server on URI /v1-rke2/client-kubelet.crt. The server will handle this request and generates a cert and key signed by client-ca.crt

client-kube-proxy.crt/client-kube-proxy.key (to be used in kubeproxy’s kubeconfig)

A request is initiated from the agent to the server on URI /v1-rke2/client-kubeproxy.crt. The server will handle this request and generates a cert and key signed by client-ca.crt

client-rke2-controller.crt/client-rke2-controller.key

A request is initiated from the agent to the server on URI /v1-rke2/client-rke2-controller.crt. The server will handle this request and returns back the cert it generated earlier for the controller

[katran001]

Brian would like to leave this issue open as it outlines an additional work part of the epic that isn't complete. The client didn't ask for it, but it's a doc that represents all of the necessary features that will be implemented at some point. Those individual features would be linked against it.

[briandowns]

This can be closed for now until the other features are asked for from a customer or otherwise needed. The initial design and implementation meets all requirements.