-
-
Notifications
You must be signed in to change notification settings - Fork 31.7k
Add socket.bind_socket() convenience function #61761
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
Here's a function similar to socket.create_connection() which addresses all the repetitive tasks needed to order to create an IPv4/IPv6 agnostic server socket. |
I think that's a good idea. However, there's a problem with the implementation: if one passes "" or None as address on a dual-stack node, the resulting socket will be either IPv4 bound to INADDR_ANY, or IPv6 bound to IN6ADDR_ANY, whereas one would expect to be bound both in IPv4 and IPv6. In the later case, some platforms (like Linux) use IPv4-mapped addresses by default, some others (e.g. some versions of FreeBSD) don't. So, depending on IPV6_V6ONLY setting, binding to IPV6 any won't accept IPv4 connections. Also, some platforms don't support mapped addresses at all, so the only portable solution is to bind both to 0.0.0.0 and ::, whith two different sockets, and use select() to accept both. So it would maybe make sense to expose a ServerSocket class, with an accept method which would do the right thing (as an added bonus, it could expose set_reuse_addr(bool), since SO_REUSEADDR have subtle semantic differences between Unix and Windows). |
What you say is right but whether the kernel supports an hybrid IPv4/6 stack or not there's not much we can do about it anyway.
...on Linux will create a socket which is reachable both as "::1" and "127.0.0.1". |
Side note: this is how in pyftpdlib I determine whether a platform supports the dual stack: def support_hybrid_ip_v4_v6():
# Note: IPPROTO_IPV6 constant is broken on Windows, see:
# http://bugs.python.org/issue6926
sock = None
try:
if not socket.has_ipv6:
return False
sock = socket.socket(socket.AF_INET6)
return not sock.getsockopt(socket.IPPROTO_IPV6, socket.IPV6_V6ONLY)
except (socket.error, AttributeError):
return False
finally:
if sock is not None:
sock.close() |
Tulip has something similar. Someone should compare the two and make sure they are equivalent or similar. |
There's a confusion.
Try the same thing after: It won't accept IPv4 connections anymore. And that's the default on many (most) platforms, e.g. FreeBSD and So the bottom line is that with the current code, on some - most - The proper way to procedd, on platforms which don't support unsetting This would propably belong to an overriden accept() method in a |
Thanks for clarifying, I have a better understanding of the problem now. |
Indeed. So we might opt for a best-effort approach: try disabling IPV6_V6ONLY |
Agreed. Then it probably makes sense to expose also a socket.has_dual_stack() function so that the user can pre-emptively decide whether using a custom class. |
I managed to write a container class which listens on multiples addresses and uses select/poll on accept() as suggested by Charles. It is available as a recipe here: IMO has_dual_stack() and create_server_sock() functions can already be included as-is (will adapt the recipe in order to get rid of Python 2.X support and submit another patch). As for including also MultipleSocketsListener that's debatable. |
Perhaps you can contribute something like this to Tulip? We've got code to run a server that can handle IPv4 and IPv6, but we currently don't have something that just creates a separate socket for each address family. Our UDP and TCP paths are also quite different. |
Yep, no prob. It would also be a good chance to test it in a real-world app. Where should I look? |
Tulip is at code.google.com/p/tulip On Thu, Mar 28, 2013 at 7:51 PM, Giampaolo Rodola'
|
I meant in the code (and what needs to be done/refactored exactly). |
start_serving() in base_events.py. On Fri, Mar 29, 2013 at 4:58 AM, Giampaolo Rodola'
|
Being Tulip asynchronous I think that what it needs is an utility function which returns *multiple* sockets as are the addresses returned by getaddrinfo() and also possibly even disable the IPv4/6 dual stack in order to be consistent across all platforms. After the sockets are returned they can be "registered" against the event loop as two separate entities such as, say, ("0.0.0.0", 8000) *and* ("::", 8000). My current recipe is different in that it provides a function which bind()s on one socket only and tries to enable the dual stack whenever possible in order to support IPv4 and IPv6 with a single socket. |
Nikolay, you may want to check out this Python tracker issue. Giampaolo: I didn't even know it was possible for a single socket to be On Mon, Apr 1, 2013 at 1:13 PM, Giampaolo Rodola' <report@bugs.python.org>wrote:
|
Here's a patch for Tulip: |
Since Tulip/asyncio has gone through a lot of development since this issue was added, I wasn't sure if this has been included already or if there was still interest in it. In either case, I think it might be able to be closed, but I wanted to make sure first. Thanks! |
Interesting. Yes, I agree this proposal is still desirable. We can reuse socket.create_server_sock() in smtpd, ftplib, socketserver (bpo-20215) and http.server (bpo-24209) modules which will inherit dual-stack IPv4/6 capabilities for free. I should be able to provide a PR sometime during this month. |
In bpo-24209, I ended up settling on this implementation ( Lines 1227 to 1234 in f289084
|
After careful thinking I realize I'm not completely sure about how to expose the IPv4/6 functionality yet. I prefer to defer it for later and start landing a bind_socket() utility function which serves as a base for this functionality. See: bpo-17561. |
After iterating over this over the last few days I realized it makes more sense to implement and discuss the whole thing in here and as a single PR, so sorry about previously splitting this in a separate ticket/PR. Relevant PR is now this one and is ready for review:
Also, I'm CC-ing people from bpo-20215 as it contains relevant comments. |
Patch committed as of: # IPv4 only
>>> socket.create_server(addr)
# IPv6 only
>>> socket.create_server(addr, family=socket.AF_INET6)
# IPv4 + IPv6
>>> socket.create_server(addr, family=socket.AF_INET6, dualstack_ipv6=True)
# IPv4/6 if possible and don't care about IPv4 mapped addresses, else IPv4
>>> if socket.has_dualstack_ipv6():
... s = socket.create_server(addr, family=socket.AF_INET6, dualstack_ipv6=True)
... else:
... s = socket.create_server(addr) |
The change broke multiple buildbots. Example: https://buildbot.python.org/all/#builders/16/builds/2625 Traceback (most recent call last):
File "/home/dje/cpython-buildarea/3.x.edelsohn-sles-z/build/Lib/test/_test_multiprocessing.py", line 4377, in test_wait_socket
self.assertEqual(b''.join(v), expected)
AssertionError: b'1\n2\n3\n4\n5\n6\n7\n8\n9\n' != b'0\n1\n2\n3\n4\n5\n6\n7\n8\n9\n' More logs: |
Fixed. There's a remaining failing BB: |
""" That's known and unrelated issue: https://bugs.python.org/issue31453 |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: