Pulling certificates from external store and only renewing if expired

[majolo]

We are working on a product that involves spinning up Kubernetes clusters and each one gets issued with a subdomain and associated certificate by LetsEncrypt. However these clusters can be deleted and recreated multiple times a week, issuing the same cert each time (subdomain.example.com) and we can possibly run foul of LetsEncrypt's 5 renewals/week rate limit.

Since these certs haven't expired (they're just being thrown away on cluster deletion) it would make sense to instead store them somewhere. I am envisaging a solution something along the lines of:

• Cert-manager tries to pull a certificate from an external store
• If it succeeds: a secret is created and all is good
• If the certificate is expired or non-existent, cert-manager uses the issuer to issue a new certificate and pushes it to the external store

I'd like to avoid reinventing the wheel if possible, does a solution already exist for something like this?

[maelvls]

I assume this would mean somehow backing up both the private key and certificate and restoring them. Adding this feature in cert-manager itself would be tricky since it is a very specific use case.

One idea could be to turn off the built-in key manager controller with:

--controllers=\*,certificates-key-manager

and then re-implement the key manager by taking inspiration from keymanager_controller.go, maybe something like this (inspired from the diagram Certificate Lifecycle):

Source: custom-keymanager.drawio.zip

I have never tried to replace one of the built-in controllers, but I'd definitely be interested to know how it would look like!!

[maelvls]

Another idea brought up by @wallrj is to use the fact that you can pre-provision a certificate Secret and cert-manager will just use it without renewing it if it is not to be renewed yet. So maybe backing up and then restoring Secret resources when spinning up the new Kubernetes cluster would work?

Related:

• Explain that you can pre-provision a Secret and Certificate.Spec.SecretName can refer to an existing Secret website#776

[majolo]

Thanks for your answers @maelvls and sorry for being so slow to reply. Yes this is roughly how I envision the solution might look like. Right now we're re-architecting to work around the rate limits but if we do end up implementing something like this I'll try to remember to post back here in case anyone else can get value from it.