curl hangs on h2c GET #192

Closed
icing opened this Issue Apr 1, 2015 · 9 comments

Projects

None yet

3 participants

@icing
icing commented Apr 1, 2015

Testing mod_h2 with curl reduced my overall hair count over the last two days. Fortunately, it also made me learn things about tcpdump and wireshark,

The culprits:
curl 7.42.0-DEV (x86_64-unknown-linux-gnu) libcurl/7.42.0-DEV OpenSSL/1.0.1f zlib/1.2.8 nghttp2/0.7.9 and
curl 7.41.0 (x86_64-apple-darwin14.1.0) libcurl/7.41.0 OpenSSL/1.0.2a zlib/1.2.8 nghttp2/0.7.9
with command line:
curl -v --http2 http://10.211.55.19:12345/index.html from osx to ubuntu and
curl -v --http2 http://10.211.55.2:12345/index.html from ubuntu to osx.

The client on OSX works fine as show in this wireshark shot:
osx2ubuntu

The client on Ubuntu however does not send anything after the HTTP2 preface. Just silent, no ACK for the SETTINGS from the server:
ubuntu2osx

At this point the connection just hangs as the server is unwilling to send the response before an ACK to the SETTINGS. Is there a flush missing somewhere maybe?

@bagder bagder added the HTTP/2 label Apr 1, 2015
@bagder bagder self-assigned this Apr 1, 2015
@bagder
Member
bagder commented Apr 1, 2015

Thanks! Can you repeat this problem against a public URL or do we need to run a local version of mod_h2?

@icing
icing commented Apr 1, 2015

One of these days I have to setup a public mod_h2 installation, but currently I do not have one. You could try to build the sandbox or wait until you are back from vacation. Now that I know the nature of the problem, I just disabled the affected test cases and can continue working.

@tatsuhiro-t
Contributor

I tried mod_h2-0.4.2 with latest curl + nghttp2 both on debian machine, upgrade worked fine.
So probably, network between two machine or mac OS X related problem?

@icing
icing commented Apr 1, 2015

Well, unfortunately, it seems not so simple. I had the effect also when the client and server ran on the same ubuntu image. Only tcpdump did not work so nicely then. The problem was originally reported by another dev working on another ubuntu image.

It seems timing sensitive. The slower the machine (less CPU core, more detailed logging) the more often it happens. That is why I suspected some flush behaviour where the ACK frame lingers in some buffer somewhere....

The nghttp client always works in all my configurations. It's really the SETTINGS+ACK from the client to the server that is not happening and everything stalls. Or so I read it. I was on it the whole yesterday and today and my brain is a bit foggy...

@tatsuhiro-t
Contributor

You are right, this is timing dependent.
While I'm reading curl code, I probably found the cause.
It is possible that curl does not send SETTINGS ACK if SETTINGS frame is received before curl sends MAGIC. We have missing nghttp2_session_send() call in Curl_http2_switched().
I'll make a PR for this soon... Could you verify that whether it works?

@icing
icing commented Apr 1, 2015

Tatsuhiro-sama, that's it! The SETTINGS is indeed sent before the preface arrives. Champagne! :-D

Am 01.04.2015 um 17:25 schrieb Tatsuhiro Tsujikawa notifications@github.com:

You are right, this is timing dependent.
While I'm reading curl code, I probably found the cause.
It is possible that curl does not send SETTINGS ACK if SETTINGS frame is received before curl sends MAGIC. We have missing nghttp2_session_send() call in Curl_http2_switched().
I'll make a PR for this soon... Could you verify that whether it works?


Reply to this email directly or view it on GitHub.

bytes GmbH
Hafenweg 16, 48155 Münster, Germany
Phone: +49 251 2807760. Amtsgericht Münster: HRB5782

@tatsuhiro-t
Contributor

PR #194
I hope it works..

@icing
icing commented Apr 2, 2015

It works like a dream. All go for pull from my side.

@bagder bagder added a commit that referenced this issue Apr 2, 2015
@tatsuhiro-t @bagder tatsuhiro-t + bagder http2: Fix missing nghttp2_session_send call in Curl_http2_switched
Previously in Curl_http2_switched, we called nghttp2_session_mem_recv to
parse incoming data which were already received while curl was handling
upgrade.  But we didn't call nghttp2_session_send, and it led to make
curl not send any response to the received frames.  Most likely, we
received SETTINGS from server at this point, so we missed opportunity to
send SETTINGS + ACK.  This commit adds missing nghttp2_session_send call
in Curl_http2_switched to fix this issue.

Bug: #192
Reported-by: Stefan Eissing
21e82bd
@bagder
Member
bagder commented Apr 2, 2015

PR #194 has been merged, case closed!

@bagder bagder closed this Apr 2, 2015
@jgsogo jgsogo added a commit to jgsogo/curl that referenced this issue Oct 19, 2015
@tatsuhiro-t @jgsogo tatsuhiro-t + jgsogo http2: Fix missing nghttp2_session_send call in Curl_http2_switched
Previously in Curl_http2_switched, we called nghttp2_session_mem_recv to
parse incoming data which were already received while curl was handling
upgrade.  But we didn't call nghttp2_session_send, and it led to make
curl not send any response to the received frames.  Most likely, we
received SETTINGS from server at this point, so we missed opportunity to
send SETTINGS + ACK.  This commit adds missing nghttp2_session_send call
in Curl_http2_switched to fix this issue.

Bug: curl#192
Reported-by: Stefan Eissing
6f3a0a3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment