checkDNSPropagation does not handle NS delegation - blocked by checkAuthoritativeNss

[cnbeining]

Welcome

• Yes, I'm using a binary release within 2 latest releases.
• Yes, I've searched similar issues on GitHub and didn't find any.
• Yes, I've included all information below (version, config, etc).

What did you expect to see?

lego able to handle DNS-01 challenge via NS delegation.

In not-so-human-language -

checkDNSPropagation should consider that, if _acme-challenge.{domain} record is setup with NS delegation, checkAuthoritativeNss will always fail but this is a legitimate use case.

NS delegation in human language:

• Set {domain}'s aDNS to {NS1}
• On {NS1}, add NS record: _acme-challenge.{domain} to {NS2}

because maybe the user cannot control all records on {NS1}, or cannot control automatically.

In this case, if {NS2} works properly, aka setting up correct record for _acme-challenge.{domain} as per requested by ACME service -

• dig _acme-challenge.{domain} @{NS2} TXT will work
• dig _acme-challenge.{domain} @{any 3rd party rDNS} TXT will work
• dig _acme-challenge.{domain} @{NS1} TXT will ALWAYS NOT WORK since there's no such record on {NS1}, and the record will not be passed on automatically as what usually happens with CNAME delegation
• dig _acme-challenge.{domain} @{NS1} NS will return {NS2}

Background reading: https://www.eff.org/deeplinks/2018/02/technical-deep-dive-securing-automation-acme-dns-challenge-validation

Proposed fixes:

1. Make checkAuthoritativeNss WARN only and let ACME provider try its luck since dnsQuery(fqdn, dns.TypeTXT, recursiveNameservers, true) works - probably dangerous to implement;
2. Expose requireCompletePropagation option - more mental burden for users;
3. Change the logic that determines aDNS in checkDNSPropagation at L70, authoritativeNss, err := lookupNameservers(fqdn): in lookupNameservers, add logic to prefer NS record of _acme-challenge.{fqdn} than NS record of {fqdn} to handle NS delegation. Need to test whether this approach works with CNAME delegation.

What did you see instead?

2022/02/21 05:50:13 Could not obtain certificates:
    error: one or more domains had a problem:
[*.{domain}] time limit exceeded: last error: NS ns3.he.net. did not return the expected TXT record [fqdn: _acme-challenge.{domain}., value: {challenge code 1}]: 
[{domain}] time limit exceeded: last error: NS ns5.he.net. did not return the expected TXT record [fqdn: _acme-challenge.{domain}., value: {challenge code 2}]: 
Certificate generation failed.

How do you use lego?

Other

Reproduction steps

Preparation:

• Set {domain}'s aDNS to {NS1}
• On {NS1}, add NS record: _acme-challenge.{domain} to {NS2}

Testing:

• Trigger ACME process, retrieve {challenge code} from ACME provider
• On {NS2}, add TXT record _acme-challenge.{domain} as the challenge code
• A dig _acme-challenge.{domain} TXT @8.8.8.8 or any rDNS correctly shows the TXT record on {NS2}.
• lego refuse to continue with verification:

2022/02/21 05:50:11 [INFO] [{domain}] acme: Cleaning DNS-01 challenge
2022/02/21 05:50:13 [INFO] retry due to: acme: error: 400 :: POST :: https://acme-v02.api.letsencrypt.org/acme/authz-v3/80597452920 :: urn:ietf:params:acme:error:badNonce :: JWS has an invalid anti-replay nonce: "{challenge code} "
2022/02/21 05:50:13 [INFO] Deactivating auth: https://acme-v02.api.letsencrypt.org/acme/authz-v3/80597452920
2022/02/21 05:50:13 [INFO] Deactivating auth: https://acme-v02.api.letsencrypt.org/acme/authz-v3/80597452930
2022/02/21 05:50:13 Could not obtain certificates:
	error: one or more domains had a problem:
[{domain}] time limit exceeded: last error: NS <NS1>. did not return the expected TXT record [fqdn: _acme-challenge.{domain}, value:{challenge code} ]: 

Version of lego

Unable to produce - sorry :-( 

Using DirectAdmin - unfortunately not admin.

Logs

Found wildcard domain name and http challenge type, switching to dns-01 validation.
2022/02/21 05:39:02 [INFO] [{domain}, *.{domain}] acme: Obtaining SAN certificate
2022/02/21 05:39:03 [INFO] [*.{domain}] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/80597452920
2022/02/21 05:39:03 [INFO] [{domain}] AuthURL: https://acme-v02.api.letsencrypt.org/acme/authz-v3/80597452930
2022/02/21 05:39:03 [INFO] [*.{domain}] acme: use dns-01 solver
2022/02/21 05:39:03 [INFO] [{domain}] acme: Could not find solver for: tls-alpn-01
2022/02/21 05:39:03 [INFO] [{domain}] acme: Could not find solver for: http-01
2022/02/21 05:39:03 [INFO] [{domain}] acme: use dns-01 solver
2022/02/21 05:39:03 [INFO] [*.{domain}] acme: Preparing to solve DNS-01
2022/02/21 05:39:06 [INFO] [*.{domain}] acme: Trying to solve DNS-01
2022/02/21 05:39:06 [INFO] [*.{domain}] acme: Checking DNS record propagation using [8.8.8.8:53]
2022/02/21 05:39:36 [INFO] Wait for propagation [timeout: 5m0s, interval: 30s]
2022/02/21 05:39:36 [INFO] [*.{domain}] acme: Waiting for DNS record propagation.
2022/02/21 05:40:06 [INFO] [*.{domain}] acme: Waiting for DNS record propagation.
2022/02/21 05:40:36 [INFO] [*.{domain}] acme: Waiting for DNS record propagation.
2022/02/21 05:41:06 [INFO] [*.{domain}] acme: Waiting for DNS record propagation.
2022/02/21 05:41:37 [INFO] [*.{domain}] acme: Waiting for DNS record propagation.
2022/02/21 05:42:07 [INFO] [*.{domain}] acme: Waiting for DNS record propagation.
2022/02/21 05:42:37 [INFO] [*.{domain}] acme: Waiting for DNS record propagation.
2022/02/21 05:43:07 [INFO] [*.{domain}] acme: Waiting for DNS record propagation.
2022/02/21 05:43:37 [INFO] [*.{domain}] acme: Waiting for DNS record propagation.
2022/02/21 05:44:07 [INFO] [*.{domain}] acme: Waiting for DNS record propagation.
2022/02/21 05:44:37 [INFO] [*.{domain}] acme: Cleaning DNS-01 challenge
2022/02/21 05:44:38 [INFO] [{domain}] acme: Preparing to solve DNS-01
2022/02/21 05:44:41 [INFO] [{domain}] acme: Trying to solve DNS-01
2022/02/21 05:44:41 [INFO] [{domain}] acme: Checking DNS record propagation using [8.8.8.8:53]
2022/02/21 05:45:11 [INFO] Wait for propagation [timeout: 5m0s, interval: 30s]
2022/02/21 05:45:11 [INFO] [{domain}] acme: Waiting for DNS record propagation.
2022/02/21 05:45:41 [INFO] [{domain}] acme: Waiting for DNS record propagation.
2022/02/21 05:46:11 [INFO] [{domain}] acme: Waiting for DNS record propagation.
2022/02/21 05:46:41 [INFO] [{domain}] acme: Waiting for DNS record propagation.
2022/02/21 05:47:11 [INFO] [{domain}] acme: Waiting for DNS record propagation.
2022/02/21 05:47:41 [INFO] [{domain}] acme: Waiting for DNS record propagation.
2022/02/21 05:48:11 [INFO] [{domain}] acme: Waiting for DNS record propagation.
2022/02/21 05:48:41 [INFO] [{domain}] acme: Waiting for DNS record propagation.
2022/02/21 05:49:11 [INFO] [{domain}] acme: Waiting for DNS record propagation.
2022/02/21 05:49:41 [INFO] [{domain}] acme: Waiting for DNS record propagation.
2022/02/21 05:50:11 [INFO] [{domain}] acme: Cleaning DNS-01 challenge
2022/02/21 05:50:13 [INFO] retry due to: acme: error: 400 :: POST :: https://acme-v02.api.letsencrypt.org/acme/authz-v3/80597452920 :: urn:ietf:params:acme:error:badNonce :: JWS has an invalid anti-replay nonce: "0102KCj2DsjZYQLLjBAVkIi-eXP-PKsEXuo--3kWZs0AJBA"
2022/02/21 05:50:13 [INFO] Deactivating auth: https://acme-v02.api.letsencrypt.org/acme/authz-v3/80597452920
2022/02/21 05:50:13 [INFO] Deactivating auth: https://acme-v02.api.letsencrypt.org/acme/authz-v3/80597452930
2022/02/21 05:50:13 Could not obtain certificates:
    error: one or more domains had a problem:
[*.{domain}] time limit exceeded: last error: NS {NS1}. did not return the expected TXT record [fqdn: _acme-challenge.{domain}., value: {challenge code 1}]: 
[{domain}] time limit exceeded: last error: NS {Alternative NS1}. did not return the expected TXT record [fqdn: _acme-challenge.{domain}., value: {challenge code 2}]: 
Certificate generation failed.

Go environment (if applicable)

$ go version && go env
# paste output here

[borg1622]

I ran into the exact same issue described here.

Since the acme implementation of NixOS depends on lego I'll be glad if NS delegation would work properly.

@cnbeining In the meantime, have you found a workaround for this?
@ldez Is there any progress in fixing/implementing this?

[ldez]

I implemented the support of CNAME by default, so I think you can handle your problem by using it.