Standalone mode does not bind to ipv6

[rewbycraft]

As far as I can tell, the letsencrypt client does not bind to ipv6 in standalone mode nor do the commands given in manual mode. While this may not seem too significant, but for people with ipv6 only machines (like me) this is quite a big problem. My suggestion would be to somehow bind to both ipv4 and ipv6 at the same time.

Either way, what you're doing is a great service to the global internet community, keep on rocking!

[pde]

At the moment the server won't connect over IPv6 to perform validation, either. So this feature request is blocked (or at least not useful) until this boulder ticket is resolved.

[pde]

Subticket of #180

[rewbycraft]

While I do see where you are coming from, I have to disagree with you. Having the client listen on ipv6 is relevant right now because of NAT64.

What is NAT64, you may ask?
Basically, the network admin can set aside a block of ipv6 (the RFC range is 64:ff9b::/96) which will be 1:1 mapped to the ipv4 internet (basically, if you try to connect to the nat64 encoded version of an ipv4 it will automatically route it through to the public ipv4 (and a reverse path exists in my case)).
I am in fact in posession of an ipv4 address, it is however not directly attached to the server in question (it's behind the NAT64). Any connections to my ipv4 arrive as ipv6 on my machine. I've been able to work around the bug by using the manual option and doing some shenanigans to the given python code to make it listen on ipv6. (I can't use my web server directly due to the machine in question functioning primarily as a load balancer, as such it does not have the capability to serve a directory using an ipv6 enabled server like nginx or apache.)

At the very least, I do appreciate that you guys may look at this eventually.

[kuba]

Have you tried running the client under your setup? I don't think there is anything in the code that stops us from "binding" to IPv6. If there is, then please paste the traceback.

[rewbycraft]

There is no error, it's just not binding/listening on ipv6. Basically, it seems to be listening on 0.0.0.0 but not :: (aka, it's listening on all ipv4 ips, but not all ipv6 ips) (I gathered this information with netstat). Now, from taking a cursory glance at the code I can spot why it does this.
Your acme_standalone HTTP01Server class uses six.moves.BaseHTTPServer, which is an alias for the normal python3 class of the same name. (And thus, albeit through some layers of abstraction) is effectively doing the same thing as the commands the manual mode tells you to run. Python is strange among the higher level languages in that there are several places where it doesn't transparently handle ipv6. You have to explicitly tell it to bind to/listen on ipv6.

As an example, here is my modified version of the python SimpleHTTPServer command manual mode has you run (I added some spacing for clarity):

import BaseHTTPServer, SimpleHTTPServer, socket

SimpleHTTPServer.SimpleHTTPRequestHandler.extensions_map = {'': 'text/plain'}

class HttpServerV6(BaseHTTPServer.HTTPServer):
    address_family = socket.AF_INET6

s = HttpServerV6(('', 80), SimpleHTTPServer.SimpleHTTPRequestHandler)
s.serve_forever()

Interestingly enough, it does seem that python implements "accepting ipv4 over ipv6". What does this mean? Well, if you run this and then check netstat you'll find it's listening on :::80 (port 80 on ::), which means that technically it's only listening on ipv6. BUT ipv6 has this neat socket option called IPV6_V6ONLY (yes, I'm diving a little into the python internals here). Basically, turning this option off (which python does) allows an ipv6 socket that binds to :: (all ips/the ipv6 equivalent to 0.0.0.0) to accept both ipv6 and ipv4 connections. The only real difference here is that the ips in use are ipv6 and that ipv4 ips are "mapped" to ipv6 ips in a way similar to what NAT64 does. Although it uses ::ffff: as the prefix rather than 64:ff9b:: So both curl http://127.0.0.1/ and curl "http://\[::1\]:8080/" work (the \ is because curl is weird like that, you have to escape the [ and ] when entering ipv6 addresses).
Just for reference as to what these ips look like, here's the output I get from the script I posted above when running aformentioned curl commands:

::ffff:127.0.0.1 - - [13/Nov/2015 20:44:33] "GET / HTTP/1.1" 200 -
::1 - - [13/Nov/2015 20:45:38] "GET / HTTP/1.1" 200 -

Both calls returned the expected data. (For reference to people who've never seen ipv6 before, ::1 is the ipv6 equivalent of 127.0.0.1. Also, I need a shirt that says "No place like ::1".)

I hope this clarifies things a bit/helps.

[ghost]

I don't get why this is an "enhancement" since it can lead to all sorts of problems and weird failures on IPv6 enabled machines, like for my setup (which has perfectly working IPv4 connectivity, by the way):

1. let's encrypt is run behind a proxy which can be required for various reasons (e.g. for docker setups because tools like dockergen for automatic http routing don't like the let's encrypt simple daemon to start/quit so quickly)
2. the proxy does forward proxying to "localhost"
3. localhost resolves to ::1

Boom, broken. Would easily work if it had bound to ::0 instead of 0.0.0.0 which should be a one line change..

Edit: .. and while I could replace "localhost" with "127.0.0.1" in my proxy config, this is also a bad workaround in the long term because once let's encrypt actually supports IPv6 and uses that to reach out from the ACME server to me, I would assume it can trigger new obscure problems - so I'm basically setting myself up for future problems. Therefore, this really should be fixed in the standalone mode as soon as feasible before everyone is forced to bork their configs to work around this.

Edit 2: treat example above with care, since I just found out silly me also broke something else in my setup which is possibly the culprit. However, "localhost" does indeed resolve to "::1" on my machine. And listening to IPv6/IPv4 is pretty easy: just choosing ::0 will usually get you a dualstack socket that listens to both

[bmw]

With IPv6 support in Let's Encrypt, we should try to get this issue resolved quickly.

[schoen]

I confirmed that

from six.moves import socketserver
socketserver.TCPServer(("", 12345), socketserver.StreamRequestHandler)

does not end up listening on IPv6 interfaces. Following the comment above at #1466 (comment), apparently it might be sufficient to add

    address_family = socket.AF_INET6

at line 28 of certbot/acme/acme/standalone.py and then the servers will succeed in binding IPv6/IPv4 capable sockets. (I don't know if this is a problem on operating systems whose kernels lack any IPv6 support; maybe there's a simple way to do a try/except block that tries to create and delete an AF_INET6 socket, and then falls back to the default behavior on systems where no such socket can be instantiated?)

[schoen]

Perhaps we could do something like

    try:
        socket.socket(socket.AF_INET6)
        address_family = socket.AF_INET6
    except socket.error:
        pass

The theory is that if the system is unable to create AF_INET6 sockets at all, then we just shouldn't change the default socket type for the server class to use them. But if it can, then we should. I haven't found a non-IPv6-capable system (i.e., without IPv6 in the kernel) to try this out on!

[schoen]

To see the plausibility that this solution will work, note the error from something like socket.socket(socket.AF_IPX) :-)

[palxex]

Confirms the solution works! Without the patch my ipv6-only site cannot be reached by letscrypt server and it reports a TLS-SNI-01 error; with it everything works OK.

[HenriWahl]

Is there already a way to create or renew a certificate for the IPv6 address of a dualstack machine only even if there are A and AAAA records?
Real life example is a dyn.com dyn-DNS host which is behind a NAT-Gateway. The A-record is always the IPv4 of the gateway, the AAAA-record the real IPv6 address. certbot-auto fails due to the A-Record:

Processing /etc/letsencrypt/renewal/foo.selfip.org.conf
-------------------------------------------------------------------------------
2016-08-03 10:13:04,353:WARNING:certbot.renewal:Attempting to renew cert
from /etc/letsencrypt/renewal/foo.selfip.org.conf produced an unexpected error:
Failed authorization procedure. foo.selfip.org (tls-sni-01): urn:acme:error:unauthorized :: 
The client lacks sufficient authorization :: Incorrect validation certificate for TLS-SNI-01 challenge.
Requested 05e04a9591435fb04ff69a0a3d871b4b.134a86956fbc13ccb5467e4a29653e29.acme.invalid from 93.218.168.93:443.
Received certificate containing 'bar.homeunix.org'. Skipping.

[pde]

Does someone want to open a pull request containing the reported solution here?

[palxex]

Made one PR in #3369

[bmw]

Assuming no outside contributor picks this up, this issue probably won't make our 0.11.0 release, but I'm planning on working on this myself for 0.12.0. As of right now, those two releases are scheduled for February 1st and March 1st respectively.

[jsha]

There have been two PRs attempting this: #3369 and #3687.

#3369 was a very small fix, which set, in the server classes:

address_family = socket.AF_INET6

It had two problems: It was failing unittests, and it wouldn't work on (a) OS's that don't support IPv6 (note: different from instances that lack an IPv6 interface), and (b) Python versions compiled without IPv6.

I am still hopeful that such an approach could work. For instance, could we wrap the assignment to address_family in a try/except that just passes if there's an exception? The main blocker is that I don't know how to set up an OS without IPv6 support. I think doing that would be an important first step if someone wants to pick up this work.

[bmw]

Currently the best approach we're aware of is attempting to get both an IPv4 and an IPv6 socket, configuring the IPv6 socket to only get IPv6 traffic. There is some discussion of this in #3687.

[jsha]

Aha, I had missed this specific comment: #3687 (comment) seems to indicate an additional reason #3369's small fix wouldn't work: It would break on FreeBSD and OpenBSD based on this stackexchange answer.

[jsha]

FYI, we recently had an issue with Let's Encrypt where IPv6 validation was failing some of the time. Since we use Certbot for an end-to-end testing / monitoring tool, we might have caught this earlier if Certbot was able to handle IPv6 in standalone. So consider this a friendly request to bump priority a bit. :-)

[alexanderkjall]

Maybe a intermittent solution could be to have a parameter that tells the program to bind to IPv6, for example -6?

[jsha]

I think in general we should require a pretty high bar for adding new flags. In most cases, just binding to both is enough, and we have a plan for how to do that. So I think that's a better approach than adding a new flag.

BTW, I talked to @ohemorange offline and she's committed to implementing this soon.

[ohemorange]

Changing this to standalone because there's a separate PR out there for manual