New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow removing SAN from multidomain certificate when renewing #2071

Closed
dusek opened this Issue Jan 3, 2016 · 23 comments

Comments

@dusek

dusek commented Jan 3, 2016

When one has a multidomain certificate with many SANs (Subject Alternative Names), there can come a situation where one wants to remove certain SANs from the certificate when renewing (e.g. the removed domain is no longer used - "retired", or it has become relevant in another multidomain certificate instead; in both cases, to keep things clean, I want to remove the no longer needed SAN from the multidomain certificate).

The problem is when renewing with the reduced set of SANs, letsencrypt does not modify the existing certificate "branch", but creates a new one, usually with the -0001 suffix. That is quite understandable, as it is a potentially destructive action; however in many cases (e.g. the one I am describing above), it is not what the user wants/expects.

Example:

Multidomain certificate created and repeatedly renewed with --domains domain.tld,s1.domain.tld,s2.domain.tld,s3.domain.tld exists in /etc/letsencrypt/live/domain.tld.

Now I want to remove e.g. s3.domain.tld, so I run with --domains domain.tld,s1.domain.tld,s2.domain.tld. Since there is no way currently to specify the intention to modify the SAN list in the existing /etc/letsencrypt/live/domain.tld "branch" (is it called "lineage"?), I end up with /etc/letsencrypt/live/domain.tld (and same in archive and renewal) untouched, and instead a new "branch" ("lineage"?) created in /etc/letsencrypt/live/domain.tld-0001 with the requested domain list.

This presents a few problems:

  • I have to modify server config to point to the new certificate path (i.e. /etc/letsencrypt/live/domain.tld --> /etc/letsencrypt/live/domain.tld-0001)
  • I really don't like the "-0001" stuff optically 😊
  • it leaves /etc/letsencrypt/{live,archive,renewal}/domain.tld on disk with no further purpose

A workaround could be to (neither was tested):

  • delete /etc/letsencrypt/{live,archive,renewal}/domain.tld before doing the renewal that removes the SAN – that is a little bit disconcerting, as there will be a period where no cert will be on disk (possible downtime, if the renewal should fail, I have to restore working certificates from archive)
  • do as above (i.e. let it create the "branch" with -0001 suffix), and then somehow "rename" domain.tld-0001 to domain.tld (again, possible downtime, possible to mess up, waste of time...)
  • edit the /etc/letsencrypt/renewal/domain.tld.conf file - the domains key (and perhaps the [[webroot_map]] section) - that does not work, as letsencrypt gets the list of domains directly from the certificate itself (found it in code somewhere)

If it is of interest, there is also a relevant LE Community thread.

So I propose that letsencrypt has some means to be told that one wants to use an existing "branch" ("mileage"?) for the new certificate, even if it meant to remove some SANs from the domain.

I think from the UI point of view, for lot of cases what is decisive is the main SAN domain name, which is listed first. That often identifies "what" certificate it is, and letsencrypt already uses it for CN and for naming the "branch" ("lineage"?) on disk. In case of renewal with removing domains, the user usually uses the same CN.

So I think that a lot (if not the majority) of these cases would be covered by some option like --compress (physics opposite to "--expand" 😉), which would look whether a certificate with same CN but some SANs removed w.r.t. its current list of SANs is requested, and if yes, would update the same certificate "branch" ("lineage"?), instead of creating a new one with -0001 suffix, with the new certificate with reduced list of SANs.

Users using compress in their cron scripts or even .ini files would then just have to be advised that for this to work, they have to preserve the CN (i.e. the first domain in the domain list), and should obviously not use the same CN in more than one certificate (that could go into the documentation of the --compress option which the user would have to turn on explicitly, thus having been informed/warned).

Thanks.

@bmw

This comment has been minimized.

Show comment
Hide comment
@bmw

bmw Jan 8, 2016

Contributor

You are correct to point out our absence of --compress. This certainly should be added so I'm adding this to the milestone for our 0.3.0 release.

You may also be interested in the related #318 which is about what the client does if it fails to obtain authorization for all the names in a certificate.

Contributor

bmw commented Jan 8, 2016

You are correct to point out our absence of --compress. This certainly should be added so I'm adding this to the milestone for our 0.3.0 release.

You may also be interested in the related #318 which is about what the client does if it fails to obtain authorization for all the names in a certificate.

@bmw bmw added this to the 0.3.0 milestone Jan 8, 2016

@aptalca

This comment has been minimized.

Show comment
Hide comment
@aptalca

aptalca Jan 8, 2016

I agree with the above comments but would like to add one thing.

Compress has a different connotation (tar, gzip, etc.) so how about --shrink ?

aptalca commented Jan 8, 2016

I agree with the above comments but would like to add one thing.

Compress has a different connotation (tar, gzip, etc.) so how about --shrink ?

@dusek

This comment has been minimized.

Show comment
Hide comment
@dusek

dusek Jan 8, 2016

@aptalca you are right about that (I realized that too after submitting the issue, just did not get to add it as a comment). Of course the UI suggested by me is only to give a first shot suggestion, I am not claiming --compress is the best name, or that the command-line UI should not be even a bit different in the way it works. :-)

dusek commented Jan 8, 2016

@aptalca you are right about that (I realized that too after submitting the issue, just did not get to add it as a comment). Of course the UI suggested by me is only to give a first shot suggestion, I am not claiming --compress is the best name, or that the command-line UI should not be even a bit different in the way it works. :-)

@hultqvist

This comment has been minimized.

Show comment
Hide comment
@hultqvist

hultqvist Jan 12, 2016

I think this is a needed feature to avoid generating new folders/branch/milage/lineage when adjusting the list of domains.

Let's say you have example.com,www.example.com stored in example.com/
and example.com,www.example.com,beta.example.com stored in example.com-0001/

Now when you ask to generate a cert for example.com using the --shrink flag, which one would be updated?

hultqvist commented Jan 12, 2016

I think this is a needed feature to avoid generating new folders/branch/milage/lineage when adjusting the list of domains.

Let's say you have example.com,www.example.com stored in example.com/
and example.com,www.example.com,beta.example.com stored in example.com-0001/

Now when you ask to generate a cert for example.com using the --shrink flag, which one would be updated?

@bmw

This comment has been minimized.

Show comment
Hide comment
@bmw

bmw Jan 19, 2016

Contributor

While this doesn't solve the problem, if --shrink applies similar logic to --expand, example.com/ would be updated. This is because removing www.example.com from the cert in example.com/ would be the smallest possible change to the existing certs.

With that said, this may not be the behavior the user wants and what if there is a tie between two certs? We shouldn't just pick one arbitrarily and instead let the user decide which certificate is updated as suggested by @hultqvist in #2128.

Contributor

bmw commented Jan 19, 2016

While this doesn't solve the problem, if --shrink applies similar logic to --expand, example.com/ would be updated. This is because removing www.example.com from the cert in example.com/ would be the smallest possible change to the existing certs.

With that said, this may not be the behavior the user wants and what if there is a tie between two certs? We shouldn't just pick one arbitrarily and instead let the user decide which certificate is updated as suggested by @hultqvist in #2128.

@bmw bmw modified the milestones: 0.5.0, 0.4.0 Jan 28, 2016

@schoen

This comment has been minimized.

Show comment
Hide comment
@schoen

schoen Feb 11, 2016

Contributor

The biggest problem around this is that we don't have a way to specify particular lineages from the command line (nor is it clear what the best way to specify them would be). The code changes required would be super-tiny if we had that mechanism available. -d flags don't uniquely specify a lineage because they could match a potentially unlimited number of them!

Contributor

schoen commented Feb 11, 2016

The biggest problem around this is that we don't have a way to specify particular lineages from the command line (nor is it clear what the best way to specify them would be). The code changes required would be super-tiny if we had that mechanism available. -d flags don't uniquely specify a lineage because they could match a potentially unlimited number of them!

@bmw bmw modified the milestones: 0.6.0, 0.5.0 Feb 18, 2016

@marcan

This comment has been minimized.

Show comment
Hide comment
@marcan

marcan Feb 19, 2016

It would make sense to be able to name lineages after something other than the CN. For example, for one of my certs that handles a bunch of "misc" subdomains, any single one of them isn't really an appropriate identifier for the cert. I ended up naming it "misc.<domain>" which is now its CN and a redundant/unused name for it (it resolves since I have a DNS wildcard, but is never used), but it would've been easier if I could just name the lineage that without actually making it a part of the cert.

marcan commented Feb 19, 2016

It would make sense to be able to name lineages after something other than the CN. For example, for one of my certs that handles a bunch of "misc" subdomains, any single one of them isn't really an appropriate identifier for the cert. I ended up naming it "misc.<domain>" which is now its CN and a redundant/unused name for it (it resolves since I have a DNS wildcard, but is never used), but it would've been easier if I could just name the lineage that without actually making it a part of the cert.

@monsterbitar

This comment has been minimized.

Show comment
Hide comment
@monsterbitar

monsterbitar Mar 8, 2016

In my case, I have a handful of mail.* subdomains for various workrelated webmails in a single cert (as they go to the same actaul server, using the same webmail installation) and one of my clients used to have both their domain.net and domain.se but let their .net expire. Now my renewal fails and what to me seems like a simple solution (removing the domain and webroot information from the .conf file) is not working.

I'm not in a great hurry, I have about a month to get this fixed and assume that worst case scenario is to merely remove the certs and start over by requesting new certs.

It would've been nice to have the option of just dropping some domains from the .conf file and tell it that I have intentionally chosen to no longer have those domains in the given certificate and it would solve it from there.

monsterbitar commented Mar 8, 2016

In my case, I have a handful of mail.* subdomains for various workrelated webmails in a single cert (as they go to the same actaul server, using the same webmail installation) and one of my clients used to have both their domain.net and domain.se but let their .net expire. Now my renewal fails and what to me seems like a simple solution (removing the domain and webroot information from the .conf file) is not working.

I'm not in a great hurry, I have about a month to get this fixed and assume that worst case scenario is to merely remove the certs and start over by requesting new certs.

It would've been nice to have the option of just dropping some domains from the .conf file and tell it that I have intentionally chosen to no longer have those domains in the given certificate and it would solve it from there.

@dusek

This comment has been minimized.

Show comment
Hide comment
@dusek

dusek May 27, 2016

Just wanted to add the real scenario showing why solving this issue is important for us: on one server, we are currently using one "megacertificate" (from a different CA) for around 70 domain names. So first step in gradual migration to Let's Encrypt would be to issue the same certificate with the same 70 domain names, but this time from Let's Encrypt.

However if I wanted, as a second phase of migration to LE, to later "spin off" a certain group of domain names from this megacertificate to a certificate of its own, I would run into this issue. That is the main reason why I have not migrated yet to Let's Encrypt on that server: if I issued the megacertificate from Let's Encrypt, I could not later do this "spin off" of subsets of domain names; and I don't currently have the time to create right away all the "perfect sets" of domain names that should go into one certificate (I want to do it gradually).

The reason why we did not create those subsets already with our current CA is exactly because renewing certificates was a very demanding manual process for us, so we wanted to minimize the trouble of this process and have only one certificate to renew (and then are happy to be set for another year).

Also the reason to divide the domain name set into subsets with separate certificates is that often, these domain names have very little in common, and it simply does not feel proper to put them all together in one big certificate (sometimes even marketing reasons – to not associate two completely different websites – is relevant).

So, I am postponing migration to Let's Encrypt on that server until this issue is solved (or until I have a very clear idea about the perfect subsets of domain names, which I currently don't, and don't have time to create them at once, and even if I did, I would still like to be free to change my mind about them later, or correct a wrong "spin off").

One server where we already have Let's Encrypt deployed (and are loving it 😄 ) is one with just 3 certificates each with ~2-3 domain names, so there this issue was not a big deal.

dusek commented May 27, 2016

Just wanted to add the real scenario showing why solving this issue is important for us: on one server, we are currently using one "megacertificate" (from a different CA) for around 70 domain names. So first step in gradual migration to Let's Encrypt would be to issue the same certificate with the same 70 domain names, but this time from Let's Encrypt.

However if I wanted, as a second phase of migration to LE, to later "spin off" a certain group of domain names from this megacertificate to a certificate of its own, I would run into this issue. That is the main reason why I have not migrated yet to Let's Encrypt on that server: if I issued the megacertificate from Let's Encrypt, I could not later do this "spin off" of subsets of domain names; and I don't currently have the time to create right away all the "perfect sets" of domain names that should go into one certificate (I want to do it gradually).

The reason why we did not create those subsets already with our current CA is exactly because renewing certificates was a very demanding manual process for us, so we wanted to minimize the trouble of this process and have only one certificate to renew (and then are happy to be set for another year).

Also the reason to divide the domain name set into subsets with separate certificates is that often, these domain names have very little in common, and it simply does not feel proper to put them all together in one big certificate (sometimes even marketing reasons – to not associate two completely different websites – is relevant).

So, I am postponing migration to Let's Encrypt on that server until this issue is solved (or until I have a very clear idea about the perfect subsets of domain names, which I currently don't, and don't have time to create them at once, and even if I did, I would still like to be free to change my mind about them later, or correct a wrong "spin off").

One server where we already have Let's Encrypt deployed (and are loving it 😄 ) is one with just 3 certificates each with ~2-3 domain names, so there this issue was not a big deal.

@bmw bmw modified the milestones: 0.8.0, 0.9.0 Jun 2, 2016

@bmw bmw removed this from the 0.8.0 milestone Jun 2, 2016

@bmw bmw modified the milestones: 0.10.0, 0.9.0 Jul 7, 2016

@dwt

This comment has been minimized.

Show comment
Hide comment
@dwt

dwt Aug 8, 2016

Hi there,

I get that this issue is not solved yet - but is there a workaround that can at least fix it for now?

dwt commented Aug 8, 2016

Hi there,

I get that this issue is not solved yet - but is there a workaround that can at least fix it for now?

@dwt

This comment has been minimized.

Show comment
Hide comment
@dwt

dwt Aug 8, 2016

deleting the 0001 files and their non suffixed brethren from /etc/letsencrypt/{archive,live,renewal} seems to have done the trick as a workaround.

dwt commented Aug 8, 2016

deleting the 0001 files and their non suffixed brethren from /etc/letsencrypt/{archive,live,renewal} seems to have done the trick as a workaround.

@nerdoc

This comment has been minimized.

Show comment
Hide comment
@nerdoc

nerdoc Aug 10, 2016

But AFAIK the paths are still available in config files and broken then. Don't have any 0001 config file around now so I can't provide an example. I wrote a renewal script for using Let'sEncrypt on Uberspace.de, and dealt with that just by re-adding the new certificate after finding the new path that was just created. It's a really ugly hack.

nerdoc commented Aug 10, 2016

But AFAIK the paths are still available in config files and broken then. Don't have any 0001 config file around now so I can't provide an example. I wrote a renewal script for using Let'sEncrypt on Uberspace.de, and dealt with that just by re-adding the new certificate after finding the new path that was just created. It's a really ugly hack.

@bmw

This comment has been minimized.

Show comment
Hide comment
@bmw

bmw Aug 11, 2016

Contributor

What @dwt wrote works. Deleting /etc/letsencrypt/archive/example.com, /etc/letsencrypt/live/example.com and /etc/letsencrypy/renewal/example.com.conf will completely remove a lineage. Make sure no webserver is accessing these file paths before doing this!

Contributor

bmw commented Aug 11, 2016

What @dwt wrote works. Deleting /etc/letsencrypt/archive/example.com, /etc/letsencrypt/live/example.com and /etc/letsencrypy/renewal/example.com.conf will completely remove a lineage. Make sure no webserver is accessing these file paths before doing this!

@dwt

This comment has been minimized.

Show comment
Hide comment
@dwt

dwt Aug 16, 2016

Since this creates a new lineage, it means that the letsencrypt server thinks you have a certificate that is running out because you are not renewing it. So expect some confusing mails from letsencrypt.

dwt commented Aug 16, 2016

Since this creates a new lineage, it means that the letsencrypt server thinks you have a certificate that is running out because you are not renewing it. So expect some confusing mails from letsencrypt.

@pa-jama

This comment has been minimized.

Show comment
Hide comment
@pa-jama

pa-jama Aug 18, 2016

Facing this issue right now.

The whole system of using SANs could be improved - little documentation, and not very intuitive.

No idea why when renewing the certificate, can't just remove SANS?

pa-jama commented Aug 18, 2016

Facing this issue right now.

The whole system of using SANs could be improved - little documentation, and not very intuitive.

No idea why when renewing the certificate, can't just remove SANS?

@PaulSD

This comment has been minimized.

Show comment
Hide comment
@PaulSD

PaulSD Sep 5, 2016

Why not simply allow the user to specify the name of the renewal config file that is associated with the certificate lineage they want to remove a domain from? That seems like it would be a simple and unique way to identify the relevant certificate lineage if it can't be automatically determined from the list of domains.

Implemented here:
https://github.com/PaulSD/certbot/commit/84c9f3f8f3968d1185a233b463a05b2a1f2be294

... Or am I missing something here? (I feel like I must be missing something, because someone else would probably have implemented this same solution long ago if it were really this simple...)

PaulSD commented Sep 5, 2016

Why not simply allow the user to specify the name of the renewal config file that is associated with the certificate lineage they want to remove a domain from? That seems like it would be a simple and unique way to identify the relevant certificate lineage if it can't be automatically determined from the list of domains.

Implemented here:
https://github.com/PaulSD/certbot/commit/84c9f3f8f3968d1185a233b463a05b2a1f2be294

... Or am I missing something here? (I feel like I must be missing something, because someone else would probably have implemented this same solution long ago if it were really this simple...)

@cowlicks

This comment has been minimized.

Show comment
Hide comment
@cowlicks

cowlicks Sep 6, 2016

Contributor

@PaulSD I was reading through this issue and wondering why it wasn't done like this. Your suggestion makes sense to me.

Contributor

cowlicks commented Sep 6, 2016

@PaulSD I was reading through this issue and wondering why it wasn't done like this. Your suggestion makes sense to me.

@nerdoc

This comment has been minimized.

Show comment
Hide comment
@nerdoc

nerdoc Sep 13, 2016

To the workaround of @dwt: While removing all 0001 (etc) occurrencies cleans up the newly produced certificates, this does not help, as these are the new, valid certificates. If you delete them, you have the old one lying around, which are going to expire then. But they are used by the web server.

To be clear: deleting -0001 files doesn't help here, as the next time letsencrypt certonly is run, the -0001 files are created again. I want letsencrypt to overwrite the old ones (or give me at least the option to do that in a way).

What I need (for an automatic renewal script) is that either

  • the paths of the new certificates stay the same, even when a domain has been deleted, and no -0001 - or whatever - lineage is created, or
  • letsencrypt certonly spits out the path of the new certificates in a parseable, validated way, so I can get the path using grep/sed/whatever and point the web server to the new certificate.

I am really looking forward to a solution here... ;-)

nerdoc commented Sep 13, 2016

To the workaround of @dwt: While removing all 0001 (etc) occurrencies cleans up the newly produced certificates, this does not help, as these are the new, valid certificates. If you delete them, you have the old one lying around, which are going to expire then. But they are used by the web server.

To be clear: deleting -0001 files doesn't help here, as the next time letsencrypt certonly is run, the -0001 files are created again. I want letsencrypt to overwrite the old ones (or give me at least the option to do that in a way).

What I need (for an automatic renewal script) is that either

  • the paths of the new certificates stay the same, even when a domain has been deleted, and no -0001 - or whatever - lineage is created, or
  • letsencrypt certonly spits out the path of the new certificates in a parseable, validated way, so I can get the path using grep/sed/whatever and point the web server to the new certificate.

I am really looking forward to a solution here... ;-)

@nerdoc nerdoc referenced this issue Sep 13, 2016

Closed

switches missing #2

3 of 4 tasks complete
@yaegashi

This comment has been minimized.

Show comment
Hide comment
@yaegashi

yaegashi Oct 5, 2016

@PaulSD @cowlicks I also thought of the solution like PaulSD@84c9f3f ... just searched the certbot manual for that option today when I had many SANs to manage in a single web server. I don't see why this issue has been left unresolved for such a long time.

yaegashi commented Oct 5, 2016

@PaulSD @cowlicks I also thought of the solution like PaulSD@84c9f3f ... just searched the certbot manual for that option today when I had many SANs to manage in a single web server. I don't see why this issue has been left unresolved for such a long time.

@cowlicks

This comment has been minimized.

Show comment
Hide comment
@cowlicks

cowlicks Oct 5, 2016

Contributor

@yaegashi I can make a PR with @PaulSD 's commit tomorrow if it is working.

Contributor

cowlicks commented Oct 5, 2016

@yaegashi I can make a PR with @PaulSD 's commit tomorrow if it is working.

@airblade

This comment has been minimized.

Show comment
Hide comment
@airblade

airblade Oct 12, 2016

I have a Let's Encrypt certificate with 50 or so subdomains / SANs. I now want to renew or replace it with a certificate for the top level domain and its www subdomain only.

How can I convert the certificate? I'm happy for any kind of manual solution because once it's converted I can automate the renewal.

I've read everything above but I'm still unsure how to do it...

airblade commented Oct 12, 2016

I have a Let's Encrypt certificate with 50 or so subdomains / SANs. I now want to renew or replace it with a certificate for the top level domain and its www subdomain only.

How can I convert the certificate? I'm happy for any kind of manual solution because once it's converted I can automate the renewal.

I've read everything above but I'm still unsure how to do it...

@nerdoc

This comment has been minimized.

Show comment
Hide comment
@nerdoc

nerdoc Oct 12, 2016

What I do in such cases, if everything else fails: backup the cli.ini, remove the whole .config/letsencrypt folder, restart letsencrypt to generate the new config, and overwrite the cli.ini with the backup version, now change the domain settings there.
then run letsencrypt certonly again, and it creates a completely new certificate without all the other problems.
Then pass the certificate to the webserver.

It's a bit overkill, but works.

nerdoc commented Oct 12, 2016

What I do in such cases, if everything else fails: backup the cli.ini, remove the whole .config/letsencrypt folder, restart letsencrypt to generate the new config, and overwrite the cli.ini with the backup version, now change the domain settings there.
then run letsencrypt certonly again, and it creates a completely new certificate without all the other problems.
Then pass the certificate to the webserver.

It's a bit overkill, but works.

@ohemorange

This comment has been minimized.

Show comment
Hide comment
@ohemorange

ohemorange Oct 13, 2016

Contributor

@airblade This should be addressed in 0.10.0 with #3615. Thanks for your use case!

Contributor

ohemorange commented Oct 13, 2016

@airblade This should be addressed in 0.10.0 with #3615. Thanks for your use case!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment