New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

curl hangs on h2c GET #192

Closed
icing opened this Issue Apr 1, 2015 · 9 comments

Comments

Projects
None yet
3 participants
@icing

icing commented Apr 1, 2015

Testing mod_h2 with curl reduced my overall hair count over the last two days. Fortunately, it also made me learn things about tcpdump and wireshark,

The culprits:
curl 7.42.0-DEV (x86_64-unknown-linux-gnu) libcurl/7.42.0-DEV OpenSSL/1.0.1f zlib/1.2.8 nghttp2/0.7.9 and
curl 7.41.0 (x86_64-apple-darwin14.1.0) libcurl/7.41.0 OpenSSL/1.0.2a zlib/1.2.8 nghttp2/0.7.9
with command line:
curl -v --http2 http://10.211.55.19:12345/index.html from osx to ubuntu and
curl -v --http2 http://10.211.55.2:12345/index.html from ubuntu to osx.

The client on OSX works fine as show in this wireshark shot:
osx2ubuntu

The client on Ubuntu however does not send anything after the HTTP2 preface. Just silent, no ACK for the SETTINGS from the server:
ubuntu2osx

At this point the connection just hangs as the server is unwilling to send the response before an ACK to the SETTINGS. Is there a flush missing somewhere maybe?

@bagder bagder added the HTTP/2 label Apr 1, 2015

@bagder bagder self-assigned this Apr 1, 2015

@bagder

This comment has been minimized.

Member

bagder commented Apr 1, 2015

Thanks! Can you repeat this problem against a public URL or do we need to run a local version of mod_h2?

@icing

This comment has been minimized.

icing commented Apr 1, 2015

One of these days I have to setup a public mod_h2 installation, but currently I do not have one. You could try to build the sandbox or wait until you are back from vacation. Now that I know the nature of the problem, I just disabled the affected test cases and can continue working.

@tatsuhiro-t

This comment has been minimized.

Contributor

tatsuhiro-t commented Apr 1, 2015

I tried mod_h2-0.4.2 with latest curl + nghttp2 both on debian machine, upgrade worked fine.
So probably, network between two machine or mac OS X related problem?

@icing

This comment has been minimized.

icing commented Apr 1, 2015

Well, unfortunately, it seems not so simple. I had the effect also when the client and server ran on the same ubuntu image. Only tcpdump did not work so nicely then. The problem was originally reported by another dev working on another ubuntu image.

It seems timing sensitive. The slower the machine (less CPU core, more detailed logging) the more often it happens. That is why I suspected some flush behaviour where the ACK frame lingers in some buffer somewhere....

The nghttp client always works in all my configurations. It's really the SETTINGS+ACK from the client to the server that is not happening and everything stalls. Or so I read it. I was on it the whole yesterday and today and my brain is a bit foggy...

@tatsuhiro-t

This comment has been minimized.

Contributor

tatsuhiro-t commented Apr 1, 2015

You are right, this is timing dependent.
While I'm reading curl code, I probably found the cause.
It is possible that curl does not send SETTINGS ACK if SETTINGS frame is received before curl sends MAGIC. We have missing nghttp2_session_send() call in Curl_http2_switched().
I'll make a PR for this soon... Could you verify that whether it works?

@icing

This comment has been minimized.

icing commented Apr 1, 2015

Tatsuhiro-sama, that's it! The SETTINGS is indeed sent before the preface arrives. Champagne! :-D

Am 01.04.2015 um 17:25 schrieb Tatsuhiro Tsujikawa notifications@github.com:

You are right, this is timing dependent.
While I'm reading curl code, I probably found the cause.
It is possible that curl does not send SETTINGS ACK if SETTINGS frame is received before curl sends MAGIC. We have missing nghttp2_session_send() call in Curl_http2_switched().
I'll make a PR for this soon... Could you verify that whether it works?


Reply to this email directly or view it on GitHub.

bytes GmbH
Hafenweg 16, 48155 Münster, Germany
Phone: +49 251 2807760. Amtsgericht Münster: HRB5782

@tatsuhiro-t

This comment has been minimized.

Contributor

tatsuhiro-t commented Apr 1, 2015

PR #194
I hope it works..

@icing

This comment has been minimized.

icing commented Apr 2, 2015

It works like a dream. All go for pull from my side.

bagder added a commit that referenced this issue Apr 2, 2015

http2: Fix missing nghttp2_session_send call in Curl_http2_switched
Previously in Curl_http2_switched, we called nghttp2_session_mem_recv to
parse incoming data which were already received while curl was handling
upgrade.  But we didn't call nghttp2_session_send, and it led to make
curl not send any response to the received frames.  Most likely, we
received SETTINGS from server at this point, so we missed opportunity to
send SETTINGS + ACK.  This commit adds missing nghttp2_session_send call
in Curl_http2_switched to fix this issue.

Bug: #192
Reported-by: Stefan Eissing
@bagder

This comment has been minimized.

Member

bagder commented Apr 2, 2015

PR #194 has been merged, case closed!

@bagder bagder closed this Apr 2, 2015

jgsogo added a commit to jgsogo/curl that referenced this issue Oct 19, 2015

http2: Fix missing nghttp2_session_send call in Curl_http2_switched
Previously in Curl_http2_switched, we called nghttp2_session_mem_recv to
parse incoming data which were already received while curl was handling
upgrade.  But we didn't call nghttp2_session_send, and it led to make
curl not send any response to the received frames.  Most likely, we
received SETTINGS from server at this point, so we missed opportunity to
send SETTINGS + ACK.  This commit adds missing nghttp2_session_send call
in Curl_http2_switched to fix this issue.

Bug: curl#192
Reported-by: Stefan Eissing

@lock lock bot locked as resolved and limited conversation to collaborators May 7, 2018

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.