Issue #610 -- Tests fallback to IPv4 if IPv6 fails #611

ddriddle · 2015-05-07T19:23:43Z

On some systems binding to an IPv6 address will fail with a socket.gaierror exception even though socket.has_ipv6 is set to True. Added code to catch these exceptions and fallback to using IPv4.

shazow · 2015-05-07T19:27:58Z

@ddriddle Any ideas what conditions are required for this to happen? (ie. has_ipv6 false-positive)

@TomasTomecek Any thoughts on this? (You wrote the ipv6 detection code in our test suite before)

ddriddle · 2015-05-07T19:45:17Z

I am not sure why I am receiving a false positive for socket.has_ipv6. I am running on RHEL6. I receive this error with Python 2.6.6 and Python 2.7.5 on the same system.

Lukasa · 2015-05-07T19:56:11Z

@ddriddle Does your system support IPv6 but simply not have the module loaded?

ddriddle · 2015-05-07T21:38:03Z

I talked with my sysadmin and the system supports IPv6. The module is loaded. The problem is that localhost is an unknown host because the IPv6 address for localhost is not listed in /etc/hosts. I don't think it is a good idea to use localhost for IPv6 since even my stock Debian box does not support localhost as an alias for ::1.

If I hardcode ::1 in dummyserver/server.py then the server will start but I receive a bunch of errors from failed unit tests here is the first one.

======================================================================
ERROR: test_source_address (test.with_dummyserver.test_connectionpool.TestConnectionPool)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/services/scratch/ddriddle/src/urllib3/test/with_dummyserver/test_connectionpool.py", line 609, in test_source_address
    r = pool.request('GET', '/source_address')
  File "/services/scratch/ddriddle/src/urllib3/urllib3/request.py", line 68, in request 
    **urlopen_kw)
  File "/services/scratch/ddriddle/src/urllib3/urllib3/request.py", line 81, in request_encode_url
    return self.urlopen(method, url, **urlopen_kw)
  File "/services/scratch/ddriddle/src/urllib3/urllib3/connectionpool.py", line 597, in urlopen 
    _stacktrace=sys.exc_info()[2])
  File "/services/scratch/ddriddle/src/urllib3/urllib3/util/retry.py", line 222, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/services/scratch/ddriddle/src/urllib3/urllib3/connectionpool.py", line 544, in urlopen 
    body=body, headers=headers)
  File "/services/scratch/ddriddle/src/urllib3/urllib3/connectionpool.py", line 349, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/opt/rh/python27/root/usr/lib64/python2.7/httplib.py", line 973, in request
    self._send_request(method, url, body, headers)
  File "/opt/rh/python27/root/usr/lib64/python2.7/httplib.py", line 1007, in _send_request
    self.endheaders(body)
  File "/opt/rh/python27/root/usr/lib64/python2.7/httplib.py", line 969, in endheaders
    self._send_output(message_body)
  File "/opt/rh/python27/root/usr/lib64/python2.7/httplib.py", line 829, in _send_output
    self.send(msg)
  File "/opt/rh/python27/root/usr/lib64/python2.7/httplib.py", line 791, in send    self.connect()
  File "/services/scratch/ddriddle/src/urllib3/urllib3/connection.py", line 155, in connect
    conn = self._new_conn()
  File "/services/scratch/ddriddle/src/urllib3/urllib3/connection.py", line 134, in _new_conn
    (self.host, self.port), self.timeout, **extra_kw)
  File "/services/scratch/ddriddle/src/urllib3/urllib3/util/connection.py", line 88, in create_connection
    raise err
ProtocolError: ('Connection aborted.', gaierror(-9, 'Address family for hostname not supported'))
-------------------- >> begin captured logging << --------------------
urllib3.util.retry: DEBUG: Converted retries value: False -> Retry(total=False, connect=None, read=None, redirect=0)
urllib3.connectionpool: INFO: Starting new HTTP connection (1): localhost
--------------------- >> end captured logging << ---------------------

I will keep working on this to see if I can figure anything else out. Please tell me what you think.

shazow · 2015-05-07T23:16:33Z

I don't think it is a good idea to use localhost for IPv6 since even my stock Debian box does not support localhost as an alias for ::1.

Why is that?

Are you able to supply a different alias for your ::1? Maybe we could override the default hostname with an environment variable for the test suite if there are other people in your situation.

If I hardcode ::1 in dummyserver/server.py then the server will start but I receive a bunch of errors from failed unit tests here is the first one.

I believe that's because our code is assuming that you're passing in a hostname, rather than an IP, so it attempts to resolve it. @Lukasa @sigmavirus24 I think this is a bug? HTTP should work with an IP as a hostname, no?

sigmavirus24 · 2015-05-08T02:08:26Z

If I hardcode ::1 in dummyserver/server.py then the server will start but I receive a bunch of errors from failed unit tests here is the first one.

I don't think so @shazow. That's because the server is only starting up for IPv6 and if the tests try to use IPv4, then you'll get the gaierror that the address family we're trying to open the socket with is not supported. You would get the same error if you bound using only IPv6 and tried to open another socket connecting to it using IPv4. If the server doesn't support IPv6 or IPv4 and we try to connect using the one it doesn't support, this seems like something that we would expect to see.

Why is that?

I just spun up a Debian VM and it had 127.0.0.1 and ::1 both in /etc/hosts.

Lukasa · 2015-05-08T06:52:41Z

@sigmavirus24 is correct, /etc/hosts on a correctly-configured IPv4/6 system will have both 127.0.0.1 and ::1 as aliases for localhost. This will happen on startup if your system expects to start up with both IPv4 and IPv6 enabled.

sigmavirus24 · 2015-05-08T12:19:23Z

You can even edit it without restarting and it should work. The question should be, why does your sysadmin remove it?

Sent from my Android device with K-9 Mail. Please excuse my brevity.

ddriddle · 2015-05-11T15:01:39Z

@sigmavirus24 Does the Debian VM have an IPv6 address assigned? My Debian system does not. When I look at my /etc/hosts it has a line with ::1 hashed out. I did not change it. The difference I see is that on my Debian system if run ping6 I receive:

$ ping6 ::1
connect: Network is unreachable

But on the RHEL6 system I receive

$ ping6 ::1
PING ::1(::1) 56 data bytes
64 bytes from ::1: icmp_seq=1 ttl=64 time=0.051 ms
64 bytes from ::1: icmp_seq=2 ttl=64 time=0.067 ms
^C
--- ::1 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1452ms
rtt min/avg/max/mdev = 0.051/0.059/0.067/0.008 ms

I ran the urllib3 unit tests on my Debian system and I received the same errors as on the RHEL6 system:

$ nosetests
S.............................E.....................................S..S.............E......................Exception in thread Thread-10:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 552, in __bootstrap_inner
    self.run()
  File "/home/ddriddle/src/urllib3/dummyserver/server.py", line 88, in run
    self.server = self._start_server()
  File "/home/ddriddle/src/urllib3/dummyserver/server.py", line 75, in _start_server
    sock.bind((self.host, 0))
  File "/usr/lib/python2.7/socket.py", line 224, in meth
    return getattr(self._sock,name)(*args)
gaierror: [Errno -2] Name or service not known

^C

Note that on the Debian system socket.has_ipv6 is set to true,

$ python
Python 2.7.3 (default, Mar 14 2014, 11:57:14) 
[GCC 4.7.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import socket
>>> socket.has_ipv6
True

even though ifconfig clearly shows I have no IPv6 address:

$ ifconfig 
eth0      Link encap:Ethernet  HWaddr f0:4d:a2:dd:a6:ca  
          inet addr:192.17.24.166  Bcast:192.17.27.255  Mask:255.255.252.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:110416728 errors:0 dropped:0 overruns:0 frame:0
          TX packets:11899882 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:2729123451 (2.5 GiB)  TX bytes:2248881360 (2.0 GiB)
          Interrupt:41 Base address:0xa000 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:290421 errors:0 dropped:0 overruns:0 frame:0
          TX packets:290421 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:882914165 (842.0 MiB)  TX bytes:882914165 (842.0 MiB)

I went into my /etc/hosts file and uncommented out the lines related to IPv6. This changes the error messages but of course they still fail because IPv6 is not enabled on my Debian system.

$ nosetests
SEEEEEEEEEEException in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 552, in __bootstrap_inner
    self.run()
  File "/home/ddriddle/src/urllib3/dummyserver/server.py", line 88, in run
    self.server = self._start_server()
  File "/home/ddriddle/src/urllib3/dummyserver/server.py", line 75, in _start_server
    sock.bind((self.host, 0))
  File "/usr/lib/python2.7/socket.py", line 224, in meth
    return getattr(self._sock,name)(*args)
error: [Errno 99] Cannot assign requested address

@sigmavirus24 Can you retry your VM test but make sure the Debian system does not have an IPv6 address? Either way it is clear that socket.has_ipv6 is not reliable.

Lukasa · 2015-05-11T15:35:49Z

As to the reliability of socket.has_ipv6, the best thing to do might be to try to create an IPv6 socket and see if it fails.

ddriddle · 2015-05-11T15:50:11Z

I looked at the documentation for has_ipv6 for cPython (dev) and it states:

socket.has_ipv6

    This constant contains a boolean value which indicates if IPv6 is supported on this platform.

This is a bit cryptic but I would assume by that it means that IPv6 is supported by the system and the cPython compiler but looking at the C code it appears that is not the case. Take a look at (cpython/Modules/socketmodule.c):

#ifdef ENABLE_IPV6
    has_ipv6 = Py_True;
#else
    has_ipv6 = Py_False;
#endif
    Py_INCREF(has_ipv6);
    PyModule_AddObject(m, "has_ipv6", has_ipv6);

And the headers contain the following relevant code (cpython/Modules/socketmodule.h) :

/* VC6 is shipped with old platform headers, and does not have MSTcpIP.h
 * Separate SDKs have all the functions we want, but older ones don't have
 * any version information.
 * I use SIO_GET_MULTICAST_FILTER to detect a decent SDK.
 */
# ifdef SIO_GET_MULTICAST_FILTER
#  include <MSTcpIP.h> /* for SIO_RCVALL */
#  define HAVE_ADDRINFO
#  define HAVE_SOCKADDR_STORAGE
#  define HAVE_GETADDRINFO
#  define HAVE_GETNAMEINFO
#  define ENABLE_IPV6
# else

Looking at the headers and the C code it looks to me that has_ipv6 is set to True or False at compile time. I see no code in cPython that performs a dynamic check to set has_ipv6. What I did find in the tests is the following code (cpython/Lib/test/support/init.py):

def _is_ipv6_enabled():
    """Check whether IPv6 is enabled on this host."""
    if socket.has_ipv6:
        sock = None
        try:
            sock = socket.socket(socket.AF_INET6, socket.SOCK_STREAM)
            sock.bind((HOSTv6, 0))
            return True
        except OSError:
            pass
        finally:
            if sock:
                sock.close()
    return False

IPV6_ENABLED = _is_ipv6_enabled()

I also found the following bit of information in cPython's docs (cpython/Misc/HISTORY):

There is also a new module variable, has_ipv6, which is True iff the current Python has IPv6 support. See SF patch #658327.

Looking at the ticket it reads to me that has_ipv6 is set if cPython was compiled with IPv6 support it does not tell you if the system supports IPv6.

ddriddle · 2015-05-11T15:51:03Z

@Lukasa completely agree. That is what this patches does.

On systems with IPv6 disabled all IPv6 unit tests will now be skipped, instead of throwing an exception. On some IPv6 enabled systems with badly configured DNS, where the IPv6 loopback address ::1 is not assigned to localhost, the unit tests will fallback to using IPv4 for tests that bind to localhost instead of throwing an socket.gaierror exception. IPv6 tests that bind directly to ::1 will continue to be run.

ddriddle · 2015-05-12T14:50:09Z

I made changes to the patch and added comments now that I better understand the problem. When you have a chance please review it and tell me if it needs any further improvements.

Tests fallback to IPv4 if IPv6 fails

shazow · 2015-05-12T16:59:30Z

Thanks @ddriddle!

ddriddle mentioned this pull request May 7, 2015

Unit tests fail with exception gaierror #610

Closed

ddriddle force-pushed the issue_610 branch from e40a402 to cf1dd49 Compare May 11, 2015 20:47

shazow added a commit that referenced this pull request May 12, 2015

Merge pull request #611 from ddriddle/issue_610

4180af1

Tests fallback to IPv4 if IPv6 fails

shazow merged commit 4180af1 into urllib3:master May 12, 2015

ncoghlan mentioned this pull request Dec 13, 2016

localhost DNS resolution problem in tests with misconfigured /etc/hosts #1070

Closed

deltaclock mentioned this pull request Dec 15, 2022

Update webconfig.py to correctly detect IPv6 support. fish-shell/fish-shell#9416

Closed

3 tasks

richard-to mentioned this pull request Jun 22, 2024

Use ipv4 host if ipv6 is not available google/mesop#497

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Issue #610 -- Tests fallback to IPv4 if IPv6 fails #611

Issue #610 -- Tests fallback to IPv4 if IPv6 fails #611

ddriddle commented May 7, 2015

shazow commented May 7, 2015

ddriddle commented May 7, 2015

Lukasa commented May 7, 2015

ddriddle commented May 7, 2015

shazow commented May 7, 2015

sigmavirus24 commented May 8, 2015

Lukasa commented May 8, 2015

sigmavirus24 commented May 8, 2015

ddriddle commented May 11, 2015

Lukasa commented May 11, 2015

ddriddle commented May 11, 2015

ddriddle commented May 11, 2015

ddriddle commented May 12, 2015

shazow commented May 12, 2015

Issue #610 -- Tests fallback to IPv4 if IPv6 fails #611

Issue #610 -- Tests fallback to IPv4 if IPv6 fails #611

Conversation

ddriddle commented May 7, 2015

shazow commented May 7, 2015

ddriddle commented May 7, 2015

Lukasa commented May 7, 2015

ddriddle commented May 7, 2015

shazow commented May 7, 2015

sigmavirus24 commented May 8, 2015

Lukasa commented May 8, 2015

sigmavirus24 commented May 8, 2015

You can even edit it without restarting and it should work. The question should be, why does your sysadmin remove it?

ddriddle commented May 11, 2015

Lukasa commented May 11, 2015

ddriddle commented May 11, 2015

ddriddle commented May 11, 2015

ddriddle commented May 12, 2015

shazow commented May 12, 2015