New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: Let's encrypt enhancements #457

Closed
dphi opened this Issue Jul 1, 2017 · 10 comments

Comments

Projects
None yet
5 participants
@dphi

dphi commented Jul 1, 2017

Hi,

I know that LE is still in beta mode. I've just batch-enabled LE for a couple of customers and ran into a few edge cases of the current implementation:

  • each customer has a le account. There's a rate-limit of adding 10 accounts per three hours.
    Proposal: Only use one LE account per Froxlor site. The one for the Froxlor domain should be used. (This is also the suggestion from LE.)

  • each "domain" in froxlor gets an own certificate. With many sub-domains this easily does not work with the current limits. Proposal: Aggregate domains per customer, ie, create *.domain.tld certificates.

  • all standard subdomains should also be aggregated to a single certificate.

I anticipate the following changes for this:

  • Update cron job to use system LE account always
  • Remove per customer LE accounts from db
  • Update DB structure to map froxlor domains to certificates

Comments?

@nachtgeist

This comment has been minimized.

Show comment
Hide comment
@nachtgeist

nachtgeist Jul 1, 2017

Contributor

You said it yourself: you ran into corner cases.

  1. This will not happen regularly, except for bulk-enabling LE on a fresh installation or if you receive a customer order for a larger number of domains.

  2. Definitely a no-go. A customer might be hosting domains for several different organizations, which, from a public point of view, she might not want to appear connected to each other. I might think about consolidating certificates per 2nd-level-domain, though. On a side note: consolidation is already being done for aliased domains.

  3. see above

As for the required changes: implementing consolidation on a 2nd-level-domain basis is trickier than it might appear on the first glance. Any change WRT the SSL-status (off/provided cert/LE-cert) of any of the domains contemplatable for consolidation has to trigger appropriate CSRs (i.e. an additional domain joining/leaving a consolidation group, creation/removal of domains and many more...)

Long story short: this will not receive high priority - at least not if I'm to implement it. As for pull requests: I'm currently swamped, so even a review might take weeks.

Contributor

nachtgeist commented Jul 1, 2017

You said it yourself: you ran into corner cases.

  1. This will not happen regularly, except for bulk-enabling LE on a fresh installation or if you receive a customer order for a larger number of domains.

  2. Definitely a no-go. A customer might be hosting domains for several different organizations, which, from a public point of view, she might not want to appear connected to each other. I might think about consolidating certificates per 2nd-level-domain, though. On a side note: consolidation is already being done for aliased domains.

  3. see above

As for the required changes: implementing consolidation on a 2nd-level-domain basis is trickier than it might appear on the first glance. Any change WRT the SSL-status (off/provided cert/LE-cert) of any of the domains contemplatable for consolidation has to trigger appropriate CSRs (i.e. an additional domain joining/leaving a consolidation group, creation/removal of domains and many more...)

Long story short: this will not receive high priority - at least not if I'm to implement it. As for pull requests: I'm currently swamped, so even a review might take weeks.

@dphi

This comment has been minimized.

Show comment
Hide comment
@dphi

dphi Jul 2, 2017

Here's my patch for 1): #458

@nachtgeist; Do you mean by 2nd level domain to essentially create *.domain.tld certificates per froxlor customer? This was my original proposal.

About; 3) I see the issue with the standard sub-domains. My solution right now is to deactivate SSL for those domains to avoid the rate limits. I'd propose to enable https by default with a shared certificate for all cutomers; but only for the standard sub-domains.

The only alternative that I see is to implement rate limit avoidance...

dphi commented Jul 2, 2017

Here's my patch for 1): #458

@nachtgeist; Do you mean by 2nd level domain to essentially create *.domain.tld certificates per froxlor customer? This was my original proposal.

About; 3) I see the issue with the standard sub-domains. My solution right now is to deactivate SSL for those domains to avoid the rate limits. I'd propose to enable https by default with a shared certificate for all cutomers; but only for the standard sub-domains.

The only alternative that I see is to implement rate limit avoidance...

@nachtgeist

This comment has been minimized.

Show comment
Hide comment
@nachtgeist

nachtgeist Jul 2, 2017

Contributor
  1. I just read through https://letsencrypt.org/docs/rate-limits/ and due to

We recently (April 2017) introduced a Failed Validation limit of 5 failures per account, per hostname, per hour. This limit will be higher on staging so you can use staging to debug connectivity problems.

I'm inclined to reject #458 . If there's a configuration error which prevents domain validation, this would mean that your entire customer base for which renewals are due is hung out to dry, not just the one domain failing validation. What's your take on this, @d00p ?

  1. Ok, after re-reading your initial post, we're on the same page.

  2. Privacy concerns still apply. But let's keep this technical: what's the exact limit you're running into here? Cert renewals are exempted from rate-limiting, so AFAICS unless you're acquiring 20 new domains per week you should be fine, once all of your domains got processed (and some waited out the rate-limiting...) after bulk-enabling LE. Or am I missing something here?

And finally: the page linked above provides a form where you can apply for an increase of rate-limits.

Contributor

nachtgeist commented Jul 2, 2017

  1. I just read through https://letsencrypt.org/docs/rate-limits/ and due to

We recently (April 2017) introduced a Failed Validation limit of 5 failures per account, per hostname, per hour. This limit will be higher on staging so you can use staging to debug connectivity problems.

I'm inclined to reject #458 . If there's a configuration error which prevents domain validation, this would mean that your entire customer base for which renewals are due is hung out to dry, not just the one domain failing validation. What's your take on this, @d00p ?

  1. Ok, after re-reading your initial post, we're on the same page.

  2. Privacy concerns still apply. But let's keep this technical: what's the exact limit you're running into here? Cert renewals are exempted from rate-limiting, so AFAICS unless you're acquiring 20 new domains per week you should be fine, once all of your domains got processed (and some waited out the rate-limiting...) after bulk-enabling LE. Or am I missing something here?

And finally: the page linked above provides a form where you can apply for an increase of rate-limits.

@dphi

This comment has been minimized.

Show comment
Hide comment
@dphi

dphi Jul 2, 2017

I don't see the problem with 1) b/c the rate limit is per "hostname", which is not the server's IP but the hostname in the certificate.

The limit I'm running into with 3) is "Certificates per Registered Domain" - which is 20 per week. I'd need for 100 standard sub-domains in froxlor 5 weeks for the initial registration. If someone has more than 240 standard sub-domains it won't be possible to register certificates for all sub-domains before the first becomes invalid...

(I don't think that asking for an exception is a good solution if it can be solved in a different way.)

dphi commented Jul 2, 2017

I don't see the problem with 1) b/c the rate limit is per "hostname", which is not the server's IP but the hostname in the certificate.

The limit I'm running into with 3) is "Certificates per Registered Domain" - which is 20 per week. I'd need for 100 standard sub-domains in froxlor 5 weeks for the initial registration. If someone has more than 240 standard sub-domains it won't be possible to register certificates for all sub-domains before the first becomes invalid...

(I don't think that asking for an exception is a good solution if it can be solved in a different way.)

@dphi

This comment has been minimized.

Show comment
Hide comment
@dphi

dphi commented Jul 18, 2017

@nachtgeist ping?

@lsbbs

This comment has been minimized.

Show comment
Hide comment
@lsbbs

lsbbs Oct 1, 2017

I would have another request for an improvement:
It is not verified if the DNS record still pointing to froxlor.
I had some domains moved to another host and they have not been disabled in froxlor. The LE certificate expired and froxlor tried to renew them. What now failed and running to the failed authentication rate limit.

lsbbs commented Oct 1, 2017

I would have another request for an improvement:
It is not verified if the DNS record still pointing to froxlor.
I had some domains moved to another host and they have not been disabled in froxlor. The LE certificate expired and froxlor tried to renew them. What now failed and running to the failed authentication rate limit.

@nachtgeist

This comment has been minimized.

Show comment
Hide comment
@nachtgeist

nachtgeist Oct 1, 2017

Contributor

It is not verified if the DNS record still pointing to froxlor.

How would you expect Froxlor to do that? If the server running Froxlor has 127.0.0.1 in its resolv.conf and only forwards queries it cannot answer itself, from Froxlor's point of view the domain's A/AAAA records will always point to itself.

[…] and they have not been disabled in froxlor

Which is what should have happened in the first place.
(If you need to keep the site up at the old server, e.g. until data migration is completed, you need to either assert the LE-cert is valid long enough to complete migration or replace it with a self-signed certificate.)
Frankly, I consider this an operator error.

Contributor

nachtgeist commented Oct 1, 2017

It is not verified if the DNS record still pointing to froxlor.

How would you expect Froxlor to do that? If the server running Froxlor has 127.0.0.1 in its resolv.conf and only forwards queries it cannot answer itself, from Froxlor's point of view the domain's A/AAAA records will always point to itself.

[…] and they have not been disabled in froxlor

Which is what should have happened in the first place.
(If you need to keep the site up at the old server, e.g. until data migration is completed, you need to either assert the LE-cert is valid long enough to complete migration or replace it with a self-signed certificate.)
Frankly, I consider this an operator error.

@lsbbs

This comment has been minimized.

Show comment
Hide comment
@lsbbs

lsbbs Oct 2, 2017

You did not get it.
A customer has a domain example.foo or created a subdomain and has LE certificates.
The domain froxlor created has the IP 1.2.3.4
Now the customer is editing the DNS . The domain is now at 5.6.7.8 but he did not delete the domain within froxlor.
When the Certificates are renewed froxlor request a renewal but LE can't get the token from .well-known/acme_challenge because it is now on a different address.
Froxlor should now simply do an DNS lookup if the domain is still under its control or if it has moved.
It happened not only once that customers miss typed IPs in the DNS editor, or moved the domain to a different hoster without telling.

lsbbs commented Oct 2, 2017

You did not get it.
A customer has a domain example.foo or created a subdomain and has LE certificates.
The domain froxlor created has the IP 1.2.3.4
Now the customer is editing the DNS . The domain is now at 5.6.7.8 but he did not delete the domain within froxlor.
When the Certificates are renewed froxlor request a renewal but LE can't get the token from .well-known/acme_challenge because it is now on a different address.
Froxlor should now simply do an DNS lookup if the domain is still under its control or if it has moved.
It happened not only once that customers miss typed IPs in the DNS editor, or moved the domain to a different hoster without telling.

@NightmareJoker2

This comment has been minimized.

Show comment
Hide comment
@NightmareJoker2

NightmareJoker2 Dec 16, 2017

@lsbbs You do not use a mere DNS lookup for this. Web servers may be behind proxies, load balancers and NAT devices. Simply perform the HTTP validation yourself. This may however not work, if DNS is resolved internally or IP traffic resolved by a NAT device, such that an internal connection would open the local site, but a request from the internet would not.

NightmareJoker2 commented Dec 16, 2017

@lsbbs You do not use a mere DNS lookup for this. Web servers may be behind proxies, load balancers and NAT devices. Simply perform the HTTP validation yourself. This may however not work, if DNS is resolved internally or IP traffic resolved by a NAT device, such that an internal connection would open the local site, but a request from the internet would not.

@d00p

This comment has been minimized.

Show comment
Hide comment
@d00p

d00p Jan 10, 2018

Member

each customer has a le account. There's a rate-limit of adding 10 accounts per three hours.
Proposal: Only use one LE account per Froxlor site. The one for the Froxlor domain should be used.
(This is also the suggestion from LE.)

this won't happen to a lot of people, very unlikeley

each "domain" in froxlor gets an own certificate. With many sub-domains this easily does not work
with the current limits. Proposal: Aggregate domains per customer, ie, create *.domain.tld certificates.

This works just fine. Also aggregating means expose all existing subdomain names, most people would not want that.Wildcard is not possible with Let's Encrypt anyway.

all standard subdomains should also be aggregated to a single certificate.

Nope, as admin, i would not want to show how many customers are on that server or what their std-subdomains are...

Member

d00p commented Jan 10, 2018

each customer has a le account. There's a rate-limit of adding 10 accounts per three hours.
Proposal: Only use one LE account per Froxlor site. The one for the Froxlor domain should be used.
(This is also the suggestion from LE.)

this won't happen to a lot of people, very unlikeley

each "domain" in froxlor gets an own certificate. With many sub-domains this easily does not work
with the current limits. Proposal: Aggregate domains per customer, ie, create *.domain.tld certificates.

This works just fine. Also aggregating means expose all existing subdomain names, most people would not want that.Wildcard is not possible with Let's Encrypt anyway.

all standard subdomains should also be aggregated to a single certificate.

Nope, as admin, i would not want to show how many customers are on that server or what their std-subdomains are...

@d00p d00p closed this Jan 10, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment