dns-rfc2136: doesn't support IPv6 DNS servers

[toreanderson]

My operating system is (include version):

Ubuntu Bionic.

I installed Certbot with (certbot-auto, OS package manager, pip, etc):

Tried first the packages bundled with Ubuntu (using apt-get install certbot). Afterwards, I tried certbot-auto (to confirm that this bug is still present in the latest upstream release).

I ran this command and it produced this output:

$ wget -O /usr/local/sbin/certbot-auto https://dl.eff.org/certbot-auto
$ chmod a+x /usr/local/sbin/certbot-auto
$ /usr/local/sbin/certbot-auto
$ cd /opt/eff.org/certbot/venv
$ source bin/activate
$ pip install certbot-dns-rfc2136
$ deactivate
$ cat <<EOF > /tmp/rfc2136.ini
dns_rfc2136_server = ipv6.google.com
dns_rfc2136_name = bogus-tsig-key
dns_rfc2136_secret = 3cMUEEgZs2tqGsyQB8X82TrK/O873NBSwkk9g2o9/VQ=
dns_rfc2136_algorithm = HMAC-SHA256
EOF
$ certbot-auto certonly --dry-run -n -m foo@bar.zo -d $(hostname -f) --dns-rfc2136 --dns-rfc2136-credentials /tmp/rfc2136.ini
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator dns-rfc2136, Installer None
Obtaining a new certificate
Performing the following challenges:
dns-01 challenge for mail2.fud.no
Unsafe permissions on credentials configuration file: /tmp/rfc2136.ini
Cleaning up challenges
Encountered exception during recovery:
Traceback (most recent call last):
  File "/opt/eff.org/certbot/venv/local/lib/python2.7/site-packages/certbot/error_handler.py", line 124, in _call_registered
    self.funcs[-1]()
  File "/opt/eff.org/certbot/venv/local/lib/python2.7/site-packages/certbot/auth_handler.py", line 220, in _cleanup_challenges
    self.auth.cleanup(achalls)
  File "/opt/eff.org/certbot/venv/local/lib/python2.7/site-packages/certbot/plugins/dns_common.py", line 77, in cleanup
    self._cleanup(domain, validation_domain_name, validation)
  File "/opt/eff.org/certbot/venv/local/lib/python2.7/site-packages/certbot_dns_rfc2136/dns_rfc2136.py", line 79, in _cleanup
    self._get_rfc2136_client().del_txt_record(validation_name, validation)
  File "/opt/eff.org/certbot/venv/local/lib/python2.7/site-packages/certbot_dns_rfc2136/dns_rfc2136.py", line 147, in del_txt_record
    domain = self._find_domain(record_name)
  File "/opt/eff.org/certbot/venv/local/lib/python2.7/site-packages/certbot_dns_rfc2136/dns_rfc2136.py", line 186, in _find_domain
    if self._query_soa(guess):
  File "/opt/eff.org/certbot/venv/local/lib/python2.7/site-packages/certbot_dns_rfc2136/dns_rfc2136.py", line 222, in _query_soa
    .format(e))
PluginError: Encountered error when making query: [Errno -2] Name or service not known
Encountered error when making query: [Errno -2] Name or service not known

Certbot's behavior differed from what I expected because:

ipv6.google.com does exist in DNS with an IN AAAA record:

$ getent ahosts ipv6.google.com
2a00:1450:400f:80b::200e STREAM ipv6.l.google.com
2a00:1450:400f:80b::200e DGRAM  
2a00:1450:400f:80b::200e RAW

If I change the DNS server to one that has both IPv4 IN A and IPv6 IN AAAA records in DNS, I do not get this error:

$ sed -i s/ipv6/www/ /tmp/rfc2136.ini
$ certbot-auto certonly --dry-run -n -m foo@bar.zo -d $(hostname -f) --dns-rfc2136 --dns-rfc2136-credentials /tmp/rfc2136.ini
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator dns-rfc2136, Installer None
Obtaining a new certificate
Performing the following challenges:
dns-01 challenge for mail2.fud.no
Unsafe permissions on credentials configuration file: /tmp/rfc2136.ini
[...]

Obviously it still doesn't work since I'm using totally bogus settings, but the key point is that the «Name or service not known» error is now gone. Using tcpdump, I can also confirm that certbot is now actually attempting to communicate with the DNS server supplied:

$ tcpdump -i any host www.google.com -n
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
15:28:02.601283 IP 10.0.0.8.51705 > 216.58.207.196.53: 7302 SOA? _acme-challenge.mail2.fud.no. (46)

Note how it is using IPv4 for this query, even though www.google.com has both IPv4 and IPv6 addresses (and so has the host certbot is running on). This violates RFC 6724, which states that IPv6 should be attempted before IPv4, if available.

Here is a Certbot log showing the issue (if available):

letsencrypt.log

[ohemorange]

Thank you for investigating and reporting this issue!

At first glance I suspect this is an issue in the dns library, although perhaps we need to call the library differently to make sure we're supporting IPv6 correctly.

But also, it looks like google is your DNS provider -- does the certbot-dns-google plugin work for your use case?

[toreanderson]

Google is not my DNS provider. I just used ipv6.google.com and www.google.com as generic replacements for IPv6-only and dual-stacked server names. It would never have worked to obtain a certificate by sending updates those hostnames, but the point of the bug is that in the ipv6.google.com case, no attempt is even made - it crashes before it gets to that point.

I could just as well have used ipv6.test-ipv6.com and ds.test-ipv6.com as generic replacements as well, as this has nothing to do with Google per se.

I did this because this issue might require me to make changes to my own DNS infrastructure (e.g., adding IPv4 IN A records) and thus make the described behaviour in the bug report no longer reproducible (if I had supplied the actual DNS server name, that is).

[noci2012]

If there is a hardcoded IPv6 address in the .ini file it does seem to work.
So the lookup in argument processing of the .ini file may fail to get from name to number.
(not investigated further).

If there is any wrong address [ valid syntax, non-existent server, server not running dns ] i only observe hanging, for both ipv4 & ipv6 trying to send the update for the _acme-challenge.
never receiving a response.
[ The name/address needs to be the one of the MASTER server btw, a slave will not do for updates, it will do for queries though ].

Maybe a query for SOA to get the correct name of the master server is needed. [ if anyone actualy does that one correctly ]. And then query this master server for the SOA record again to verify its reachability?

[osirisinferi]

The DNSPython function dns.query.tcp() accepts an address as the server argument. See the documentation:

https://dnspython.readthedocs.io/en/latest/query.html#dns.query.tcp

Currently, the dns_rfc2136_server variable from the credentials file is directly passed to the tcp() function, therefore it should also only contain addresses.

Unfortunately, this is not documented explicitely in the certbot-dns-rfc2136 documentation 😞 It is only implicitely inferred from the single example credential file.

Also, version 0.36.0 is quite a long time ago. With my currently installed certbot (1.11.0.dev0) I can't reproduce the behaviour reported here: www.google.com now also doesn't work any longer: DNSPython raises an exception:

dns.exception.SyntaxError: Text input is malformed.

Which makes sense if it expects an address in stead of a hostname.

So maybe this can be transformed into a feature request? Namely: "Add support for hostnames as DNS servers in the dns-rfc2136 credentials file in stead of only addresses."

[noci2012]

The issue is slightly more complicated though. One doesn't need to translate "just" a A/AAAA lookup. It needs to find the DNS master server for a domain. This requires the SOA record to have a valid hostname (=master DNS server) as the first parameter.
The app must be able to update that server as well.
It is probable the configuration implemented doesn't work like that. And a master server is not accessible like that.
DNS infra structure is hardly dependent on DNS names itself except from the top. Hence using an address isn't that weird.

certbot-dns-rfc2136 still works unchanged up to 1.14.0

Problem with a lot of certbot dns modules is most are not maitained in distributions, you need to do this yourself... alas.
Maybe the dns-standalone works better for you?

[osirisinferi]

One doesn't need to translate "just" a A/AAAA lookup. It needs to find the DNS master server for a domain. This requires the SOA record to have a valid hostname (=master DNS server) as the first parameter.

I don't agree with this one: the certbot-dns-rfc2136 credential documentation clearly states "Target DNS server" as explanation for the dns_rfc2136_server option. The plugin does not need to "seek" the main DNS server according to the hostname it's trying to authenticate: the user needs to input the DNS server to connect to in the credential files manually.

Now, if you'd like the certbot-dns-rfc2136 plugin to automatically infer the main DNS server from the SOA records instead of entering a server to connect to manually in the credential file, then I suggest you open a brand new feature request in a separate issue.