Add section about domain validation for wildcard domains

[diracdeltas]

Fix #64. Unfortunately a lot of substantial discussion happened in that github issue instead of on the proper mailing list (which i'll ping for review now).

I only added support for wildcard domain validation via DVSNI for now, because that seems to best fulfill the needs of everyone who has asked for wildcard certs so far.

[jsha]

What does registry-controlled mean and how do we determine which suffixes are registry-controlled?

[diracdeltas]

This was borrowed from the CAB requirement for wildcard certs. https://cabforum.org/wp-content/uploads/BRv1.2.3.pdf

It says, "Determination of what is “registry-controlled” versus the registerable portion of a Country Code Top-Level
Domain Namespace is not standardized at the time of writing and is not a property of the
DNS itself. Current best practice is to consult a “public suffix list” such as
http://publicsuffix.org/ (PSL), and to retrieve a fresh copy regularly. If using the PSL, a CA SHOULD consult the "ICANN DOMAINS" section only, not the "PRIVATE DOMAINS" section. The PSL is updated regularly to contain new gTLDs delegated by ICANN, which are listed in the "ICANN DOMAINS" section. A CA is not prohibited from issuing a Wildcard Certificate to the Registrant of an entire gTLD, provided that control of the entire namespace is demonstrated in an appropriate way."

[jsha]

So it seems like public suffixes are a superset of registry controlled ones. Let's just say public suffix here and link the PSL.

[diracdeltas]

It's confusing because the CAB baseline requirements say, "If a wildcard would fall within the label immediately to the left of a registry-controlled or public suffix, CAs MUST refuse issuance...". You'd think that if the PSL were a superset, it would just say "public suffix".

[titanous]

I have several domains on the PSL that I may want to get wildcards for, so keeping it to the ICANN domains section seems important.

[sleevi]

The PSL currently contains the IANA Root Zone Database PLUS the domain boundaries defined by registries (e.g. .co.uk). "registry controlled" covers the former, "public suffix" covers the latter.

Historically, every gTLD was implicitly a PSL, but the PSL didn't express that (it instead let it fall back to the default '*' role described in the PSL algorithm). However, as a result of implementations coming to assume that the PSL >= the RZD, it's now been automated that the RZD data feed is incorporated and expressed in the PSL.

So yes, using a single datafile will work. However, if you want more timely security, the RZD is published multiple times a day (... insecurely, naturally), or you could execute a DNS zone file transfer multiple times a day. This is because PSL updates tend to be batched every month or two.

In either event, if a gTLD is absent from the PSL, MUST NOT issue is a correct response, period. Treat it as an Intranet domain until the PSL refreshes from the RZD.

[jsha]

if a gTLD is absent from the PSL, MUST NOT issue

I think we've got this covered already here: https://github.com/letsencrypt/boulder/blob/master/policy/policy-authority.go#L124

I don't understand home fetching the RZD separately from the PSL grants more timely security. I assume deletions from RZD effectively never happen, so we would only get additions, which can only make us more permissive.

[jsha]

I believe the current Challenges implicitly contact the FQDN that the client is trying to prove control over.

We should clarify that when a client responds to the server indicating is is ready for the SimpleHTTPS or DVSNI challenges to be tested, (a) the client must be prepared to respond to any number of probes from the server, up until the server indicates acceptance of the challenge, and (b) the client must respond to all names covered by the requested indicator, which for wildcards includes both the non-wildcarded part of the domain name and all possible subdomains matching the wildcard.

We should also include a reference to the validity scope of wildcarded certificates, which I always forget. Is it one level deep or arbitrarily deep? I think the former, but let's mention and link the RFC.

[martinthomson]

It is one label only (the leaf).

[diracdeltas]

Yep, it's one label deep according to https://tools.ietf.org/html/rfc6125#section-6.4.3

[martinthomson]

I'd drop step 5. That is always true (ever if it is especially true here, that option is not unique to wildcards).

[diracdeltas]

We should clarify that when a client responds to the server indicating is is ready for the SimpleHTTPS or DVSNI challenges to be tested, (a) the client must be prepared to respond to any number of probes from the server, up until the server indicates acceptance of the challenge, and (b) the client must respond to all names covered by the requested indicator, which for wildcards includes both the non-wildcarded part of the domain name and all possible subdomains matching the wildcard.

Would this go in the wildcard section, or is it for domain validation in general?

Isn't it impossible to check for sure that the client responds to all possible subdomains matching a wildcard?

[frasertweedale]

What are the wildcard certificate use cases? Can these be made explicit in the specification?

Typically there is a small set of names used with a wildcard certificate, yet ACME is all about automating the DV. So I do not know what the use cases for wildcard certificates in ACME are. Is there another common use case for which a wildcard certificate is necessary or advantageous?

[martinthomson]

I don't think that it would be helpful to describe wildcard use cases. I don't think that there is much doubt about whether people want to have them. I've operated services that needed them and not for a small set of names (it was about 10,000 names at the time). There are a lot of wildcard certificates out there.

If people don't find that sufficiently convincing, I'm sure that we can land something in Firefox that tracks the relative rate of occurrence of wildcards. Unfortunately, we don't have telemetry for that currently.

[kentonv]

@frasertweedale One case is Sandstorm.io, a platform for personal servers. Sandstorm generates randomized subhosts to take advantage of same-origin policy for sandboxing purposes and other security reasons. There isn't really a way for Sandstorm to achieve the same security properties without generating lots of randomized hostnames, which obviously needs a wildcard cert.

[sleevi]

@kentonv Just making sure that the sandstorm.io use case doesn't benefit from ALSO getting itself on the PSL.

That is, if [randomcrapA].sandstorm.io operates in a different security zone than [randomcrapB].sandstorm.io, then you really should be adding sandstorm.io to the PSL in the private section, otherwise [randomcrapA].sandstorm.io can perform cookie pinning attacks on [randomcrapB].sandstorm.io

Which would then be doubly unfortunate, as this PR would prevent you from getting such a cert.

[kentonv]

@sleevi Sandstorm is decentralized -- many different users run many different servers at many different hostnames. Presumably it wouldn't make much sense for every one of them to try to get in the PSL. :) This is why we're interested in Let's Encrypt, of course -- we can afford a wildcard cert for our own servers but many users cannot. Pinned cookies are a challenge but the Sandstorm server should be able to do something about them (when we get around to it) since it acts as a proxy between the user and any app.

[frasertweedale]

@kentonv surely ACME itself mitigates (in principle; by way of automated DV) the need for a wildcard certificate in cases such as these. Or, put another way, is there anything specific about sandstorm.io (I am not familiar with it) that would rule out using LetsEncrypt/ACME to acquire certificates for the random subdomains.

[titanous]

This isn't actually a real DNS record, the DNS server sees it in the zone and responds to all questions that don't match any other records with an answer containing the wildcard record from the zone.

[edmonds]

No, it is a real DNS record. See RFC 4592.

[diracdeltas]

In either case, this is gonna get taken out.

[kentonv]

@frasertweedale We are talking about a new random hostname every time a user opens a document (even a document they've opened before). Requesting a new certificate would have several undesirable properties:

1. 2-5 orders of magnitude more load on Let's Encrypt's servers from Sandstorm users.
2. Latency for the user, who needs to wait for the certificate to be issued before the document they clicked on can open.
3. Revealing the random hostname to a third party compromises security somewhat. (That said, I realize DNS is plaintext; this is only defense-in-depth.)

[frasertweedale]

@kentonv fair enough, thanks for the clarification.

[jsha]

I'm convinced that it's sufficient to prove ownership of example.com in order to issue *.example.com. Perhaps rather than specifying a special way of proving ownership, we could simply say that if an account key is authorized to issue certs for the identifier example.com, they are also authorized to issue certs for *.example.com.

[diracdeltas]

@jsha Agreed, that is becoming the conclusion of these many comments. I think @pde might have a comment re: more stringent validation for wildcard domains because of phishing and other potential security risks (hence why CAs usually require human review for wildcards). But perhaps it's not necessary to have that in the ACME spec.

[geofft]

A couple of thoughts:

• It's not clear from elsewhere in the spec that the asterisk is prohibited in non-wildcard certificates. Since the asterisk will resolve in DNS and works as a hostname in at least some OSes (e.g., ping '*.github.io' works on OS X, though not on Linux), it's very important that an ACME server can't be tricked into following the non-wildcard validation rules for wildcards.
• Agreed with other comments about private domains on the PSL; it should be possible to get wildcards for those. It's fairly common to have them run by a single server operator (and thus want a single cert), but host sites for different, mutually-untrusting users. *.github.io is a good example.
  • It might be nice to have a future where you can instantly self-provision yourself onto the PSL, just like you can instantly self-provision yourself a wildcard. Could we have, say, an X.509 extension that marks the wildcard cert as public-suffix-like? That would address the concern with Sandstorm: end users could immediately get public-suffix-style isolation between their apps. (I realize this comment thread isn't the right forum for such a proposal.)
  • Regarding why provisioning individual certs is untenable: at scripts.mit.edu (where we have several thousand accounts using the *.scripts.mit.edu wildcard) we've found that Apache httpd scales incredibly poorly to having thousands of <VirtualHost> blocks, and the mass-vhosting solutions like mod_vhost_ldap don't support SNI. I'm not sure how other off-the-shelf software fares, but I'd guess not well.
• One risk in only checking example.com ownership before issuing *.example.com: scripts.mit.edu lets account names have periods in them because course numbers (like 6.001) have periods, so there are some recursion issues here. The proposed policy would potentially allow an account named 001 to (correctly) demonstrate ownership of 001.scripts.mit.edu, and (incorrectly) get a wildcard cert valid for 6.001.scripts.mit.edu. I'm not sure if we're unique here in allowing periods, but we probably aren't. Restricting wildcard validation to DVSNI probably solves this, since you have to be the server operator and not merely a web-hosting client, but I figured it's worth mentioning.

[jsha]

@geofft: We do validate identifiers as containing only legal DNS labels ([a-z0-9-]), which rules out treating the * as part of a 'regular' identifier. Not sure if this needs to be part of the spec. In general we should have a clearer line about what is part of the protocol and what is part of policy / implementation.

Regarding wildcard issuances for valid privately owned public suffixes, let me try to rephrase what you and @sleevi have said to make sure I have it right:

1. To issue for a non-wildcard FQDN, it must have a public suffix (either PRIVATE or ICANN) as a proper suffix.
2. To issue for a wildcard name, the non-wildcard part must not be exactly equal to an ICANN public suffix. The non-wildcard part must have a public suffix (PRIVATE or ICANN) as a proper suffix, OR exactly equal a PRIVATE suffix. No additional checks beyond the normal DVSNI should be required for the exactly-equal case.

[jsha]

What are people's opinions about DVSNI vs DNS TXT record as a way to prove ownership when issuing wildcard certs? DVSNI is roughly equivalent to the "put this file on your server" method at least one other CA offers for DV. Do any CAs allow the same method for wildcard certs?

It seems like provisioning a DNS TXT record potentially proves more about controlling the DNS from a subdomain, which is what's important if we're saying "control of example.com implies control of *.example.com."

[titanous]

@jsha As explained here, it'd be really helpful if DVSNI was an option for wildcards (I'm fine with the random subdomain requirement) instead of a TXT record, as modifying DNS is really tricky for many users, whereas the server/provisioning software can do the heavy lifting for DVSNI.

[geofft]

I think you just want to check that it ends with an ICANN suffix, not a PRIVATE or ICANN one. In the normal case, PRIVATE suffixes should end with ICANN ones anyway, and in the abnormal case (like if someone gets their .onion or .bit or something on the PSL, which isn't implausible), that shouldn't magically permit issuing a cert for that name. And see also the BRs: "If using the PSL, a CA SHOULD consult the 'ICANN DOMAINS' section only, not the 'PRIVATE DOMAINS' section."

So change "PRIVATE or ICANN" to just "ICANN":

1. To issue for a non-wildcard FQDN, it must have an ICANN public suffix as a proper suffix. (Or, equivalently, root zone, but the ICANN part of the PSL definitely contains only root zones and things that end in root zones.)
2. To issue for a wildcard name, the non-wildcard part must not be exactly equal to an ICANN public suffix. The non-wildcard part must have an ICANN public suffix as a proper suffix.

At that point you can refactor the must-end-with-a-real-TLD check out:

1. To issue for any name (wildcard or not), it must have an ICANN suffix as a proper suffix.
2. To issue for a wildcard, run the PSL algorithm on the name, using only the ICANN portion of the PSL. The resulting public suffix must not exactly match the non-wildcard portion of the name.

Note that explicitly introducing the PSL algorithm would prevent a wildcard from being issued for example.ck (since *.ck, not just ck, is on the PSL), which I think is correct.

I'm not sure what you meant by "No additional checks beyond the normal DVSNI should be required for the exactly-equal case." Are you intending to apply more-than-DVSNI checks for people asking for wildcards who aren't public suffixes? I'd imagine the verification procedure shouldn't change depending on whether it's a (PRIVATE) public suffix or not.

Re DVSNI vs. DNS, DNS as an additional option seems fine, but e.g. scripts.mit.edu has a *.scripts.mit.edu A record but no DNS delegation. I think it's also not uncommon for various registrars' / DNS hosts' webapps to make it much easier to add a wildcard than a TXT record. I can sorta see an argument for "DVSNI against both the portion after the wildcard, plus one name of your choosing that matches." @titanous' random subdomain requirement is certainly rigorous enough, but perhaps overly rigorous, per other comments about DNS setup. IMO, having a finite handful of names a.example.com, b.example.com, etc. is sufficient reason to want a wildcard for *.example.com; you shouldn't need to own every name.

[jhcloos]

Wildcard dns should not be a requirement for wildcard certs.

DNS can be updated as part of the deploy-a-new-name scenario w/o resorting to a wildcard, and in many cases where this has value there are other records at the same level, so a wildcard in the dns is not possible.

[kentonv]

DVSNI is roughly equivalent to the "put this file on your server" method at least one other CA offers for DV. Do any CAs allow the same method for wildcard certs?

My experience with Gandi is that they'll issue a wildcard cert based on exactly the same verification mechanisms used for regular certs on the parent domain (but 10x the price). The verification mechanisms accepted were "place specified randomly-generated file content at specified randomly-generated path on HTTP server", "create TXT record for specified randomly-generated subhost with specified randomly-generated content", and "receive e-mail at presumed administrative address" (ugh). Gandi resells Comodo/UserTrust certificates, and it appeared to me that Comodo was implementing the verification, but I don't know a lot about how these things work.

[diracdeltas]

Updated pull request based on review comments (thanks everyone):

• Remove redundant "CN or subjectAltName" wording
• Remove DNS record validation
• Clarify ICANN vs. PRIVATE domains (we check ICANN, not PRIVATE)
• Remove step 5 ("MAY perform additional validation checks...") since it's true in general

[diracdeltas]

What are people's opinions about DVSNI vs DNS TXT record as a way to prove ownership when issuing wildcard certs? DVSNI is roughly equivalent to the "put this file on your server" method at least one other CA offers for DV. Do any CAs allow the same method for wildcard certs?

Yes, there is one CA that lets you put a challenge string in a meta tag on your main page to get a wildcard cert. They also allow DNS TXT proof.

[jsha]

I think it makes more sense to change nothing at validation time: The client just validates control of the bare domain name. This logic makes more sense at issuance time.

[diracdeltas]

I think checking for domain name validity should happen at validation time. But maybe this should be put into a separate section at the start of "Identifier Validation Challenges".

[diracdeltas]

@jsha i think the current implementation is as follows:

1. Client enters the identifier they want to get validated
2. Server issues challenges
3. Client does challenges, server checks answers and issues certificate for the identifier.

The steps below would happen between (1) and (2). Are you suggesting that wildcard domain handling get pushed later?

[jsha]

1. Client enters the identifier they want to get validated.
2. Server issues challenges.
3. Client sets up challenges, tells server it's ready.
4. Server checks answers, client polls until server says they were all accepted or rejected.
5. Repeat 1-4 for all desired identifiers.

... arbitrary amount of time passes ...

1. Client POSTs /new-certificate, which contains a CSR. The CSR has a list of names in it. The server verifies that the account key is authorized for each name in the CSR.

Right now the spec doesn't go into much detail on the validation of names in the CSR, but it would be good to specify that. And that would be the place to describe wildcard domain handling, I think.

[diracdeltas]

Thanks, I see the difference. The only mention I found about name validation in the CSR is in https://github.com/letsencrypt/acme-spec/blob/master/draft-barnes-acme.md#certificate-issuance:

The values provided in the CSR are only a request, and are not guaranteed. The server or CA may alter any fields in the certificate before issuance. For example, the CA may remove identifiers that are not authorized for the key indicated in the "authorization" field.

As specified, we are only allowing DVSNI for wildcard issuance, so it makes sense to have wildcard as a special case for domain validation. Are you ok with merging this pull request as-is and opening an issue to refactor it into an expanded section about CSR name validation later?

[titanous]

The markdown renderer doesn't like this list (I think a newline above this line might fix it): https://github.com/letsencrypt/acme-spec/pull/97/files?short_path=2e5a10b#diff-2e5a10b3cfb1002733218cd4472b40a4

[diracdeltas]

Good catch; will fix.

[kyledrake]

+1. Wildcard support is going to help a lot of people.

Neocities uses a wildcard for all our domains. It costs $100/yr for the cheapest DV certificate we could find. If Let's Encrypt didn't support wildcards, I wouldn't be able to use it for our free sites, and I would be stuck paying that (or possibly much more) in the future.

[ameliaikeda]

From my experience running a CDN of radio station relays, wildcard support is literally the only thing usable for the load balancers. It's a minimum of $60 per year for this certificate, and it is the most annoying, awkward thing ever to do. I need to plan 2 weeks in advance just to actually rotate the cert out, with pinning.

CAs (Comodo, VeriSign, GeoTrust) all use simple DNS TXT records and require domain validation for the example.com portion of *.example.com, but a lot of them also ask for real-life information about the requester to store in their own internal databases.

In order to avoid what happens in the scripts.mit.edu case that @geofft mentioned, the domain itself should probably be DNS-validated to find the apex. If you control the zone, you should be able to have wildcards issued.

[hardie]

So, the typical question for wildcarding is whether you can be certain that you understand correctly where the apex is. The problem statement for the related IETF working group (dbound) is here: https://datatracker.ietf.org/doc/draft-sullivan-dbound-problem-statement/ . It refers to a bunch of other documents, including RFC 6125. My impression is that getting something that works mostly is easy but there are many more than 4 corner cases; even with a fully up-to-date public suffix list, this can be an issue.

How much of this needs to be tackled in rev 1?

[bifurcation]

I agree with @hardie that this is a difficult thing that the WG should discuss. So I'm closing this PR for now.

[techninja1008]

What's happening currently with wildcard support?

[cpu]

@techninja1008 This repository is deprecated & unmaintained. Please move discussion to the replacement IETF owned repo and the mailing list.