Join GitHub today
GitHub is home to over 20 million developers working together to host and review code, manage projects, and build software together.
HTTPClient & AsyncHTTPClient Lag/Overhead #110
I am using HTTPClient and AsyncHTTPClient to talk to a local
I just tested this with my own app (on a mac using the kqueue ioloop, libcurl 7.19.7, and AsyncHTTPClient2) and I'm not seeing this delay. I'd suggest using wireshark or tcpdump first to verify whether the delay is visible on the wire (and therefore whether the problem is client or server side).
Here I provided a server and a client which will show the issue.
I am Ubuntu 10.04
Our server has the same issue as our dev machine.
You will see an exact 40ms overhead after the first request.
This issue seems to exist in all the HTTPClients, and it only occurs after a certain amount of time it seems. If you don't put a delay between requests, they are all fast.
OK, I've been able to reproduce this with ubuntu (9.10), but I'm not sure what exactly is going on. FWIW, when I run the client and servers on different machines the problem occurs when the server is on the mac and the client is on ubuntu but not vice versa.
From tcpdump, clients on ubuntu are sending the request as two packets, one with the headers and one with the body. On the mac the entire request is sent as one packet. The delay is because the client is waiting for a tcp ack from the server before sending the second packet. It is the tcp ack from the server that is getting delayed by 30ms. I'm not sure why that would be so slow, or why it would be different after the first request on the connection.
This comment has been minimized.
This comment has been minimized.Show comment Hide comment
So this appears to be an interaction between TCP delayed ACKs and Nagle's algorithm. The server is waiting (40ms) to see if there is another packet coming before sending an ack, while the client is waiting for that ack to send the next packet. The problem goes away if I call curl.setopt(pycurl.TCP_NODELAY, 1) from a prepare_curl_callback, but I don't know if that has other undesirable consequences.