Nagle #283
Nagle #283
Conversation
|
I deleted my remote branch, that closed the old PR. This one is cleaner now |
| def _new_conn(self): | ||
| try: | ||
| conn = socket.create_connection( | ||
| (self.host,self.port), |
There was a problem hiding this comment.
PEP8 please (space after comma).
|
Looks fairly sane. Is there anything left to do here? |
|
Yeah - what do you think about making this configurable with a flag passed to the connectionpool? |
| } | ||
|
|
||
|
|
||
| class HTTPConnection(_HTTPConnection, object): |
I'm -1 to exposing yet another flag to the public interface for now, but perhaps this is a better compromise: What if we add a |
|
Ok, updated |
| self.timeout, | ||
| ) | ||
| if self.tcp_nodelay: | ||
| conn.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1) |
There was a problem hiding this comment.
If we're going to call it literally tcp_nodelay, then would it make more sense to just have 0 and 1 values, and skip the if statement with...
conn.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, self.tcp_nodelay)?
There was a problem hiding this comment.
That's not a bad idea. The advantage of naming it tcp_nodelay is it's familiar and obvious what it does to the people who want the setting. The disadvantage (I think) with that and disable_nagle is the use of a negative in the variable name, which gets doubled if you set tcp_nodelay = False.
There was a problem hiding this comment.
Agreed. Let's always explicitly set it, that way we also get the benefit of consistent behaviour despite the possibility of varying default socket options.
|
Ping @honzakral, this is relevant to your interests. :) |
|
Thanks for the ping, and for the code obviously :) It looks good. |
How are we leaving Nagle enabled for proxies? (Am I missing something?) |
|
I'm not sure what was meant by that comment. We can only disable Nagle (set socket options) in places where we actually create the socket. I grepped the source code for other references to I'm not too familiar with proxies though, it's possible I'm missing something. |
|
I think this optimization is harmful for proxies, since the underlying data transmission is less fragmented. At least if you look at other libs like httplib2, they forego disabling nagle in proxies. Perhaps what we need to do is change @honzakral Are you aware of any of the details re nagle and proxies? |
|
@shazow no, I am not aware of any differences and from the nature of things I don't understand how it would be harmful. But then again there is a reason why I am not a network programmer ;) |
|
@shazow I agree that packets can get a little smaller. I guess as a general note, it's also possible/likely people are implementing proxies outside of Python, for example connecting to HAProxy or Squid running on the same machine. I'm a little more hesitant to disable this by default... it is useful for my usecase (a time-critical API where we absolutely want packets sent as fast as possible) and @honzakral's use case as well, but not sure it is useful in the general case. Browsing other HTTP clients it seems like they generally leave it off by default. This is by no means an extensive list.
For completeness, it does look like this library sets it. |
|
Also https://code.google.com/p/httplib2/source/browse/python2/httplib2/__init__.py#886 Though the py3 version of httplib2 does not have any such thing. Thinking of it rationally, since the Nagle algorithm clumps together smaller packets into one send, it makes sense to keep it enabled for proxies, which would have all packets going to the same place, thus would have more benefit from Nagle. |
|
Sorry... picking this back up again. That's fair. I haven't worked too much with proxies and thought of them as two distinct HTTP hops, kind of like Squid, instead of a single request that is made by way of a middle server. |
|
@kevinburke Thanks for picking this up again, Kevin. Is this good to be merged on your end? |
|
Just need to figure out how to configure the setting for proxies... Kevin Burke | Twilio On Wed, Jan 15, 2014 at 5:48 PM, Andrey Petrov notifications@github.comwrote:
|
|
Pushed code that I think does what we want; I am worried by the lack of testing so I am adding tests that the socket options are set the way we expect. |
|
Could you
Excellent, thank you. Let me know when I should do a final review and merge. :) |
Nagle's algorithm can be useful in some scenarios to limit packet overhead and bandwidth over the wire. This change allows consumers to disable Nagle's algorithm by subclassing HTTPConnection in urllib3/connection.py.
|
Ok, per your comment in the other PR I changed back the names. I also updated some documentation references |
Disable Nagle's algorithm for non-proxies
|
Strange how the PR attributes these commits to me, but oh well. :) Thanks again, Kevin! |
|
Yeah I think it's cause I squashed all of my commits onto yours... I'm sure On Thursday, January 16, 2014, Andrey Petrov notifications@github.com
Kevin Burke | Twilio |
|
The squash prolly wasn't necessary, just a plain rebase would have done it. |
|
This is an important change for us, can we maybe get a release with this in it? |
|
@honzakral I've been waiting for the |
|
@honzakral Good news, v1.8 is published. :) Enjoy! |
No description provided.