Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

http3 connect failed: Resource temporarily unavailable (with ngtcp2) #6864

Closed
icebluey opened this issue Apr 7, 2021 · 10 comments
Closed

http3 connect failed: Resource temporarily unavailable (with ngtcp2) #6864

icebluey opened this issue Apr 7, 2021 · 10 comments

Comments

@icebluey
Copy link

@icebluey icebluey commented Apr 7, 2021

# curl-http3 -vI --http3 https://cloudflare-quic.com
*   Trying 2606:4700:10::6816:926:443...
* Connect socket 5 over QUIC to 2606:4700:10::6816:926:443
* connect to 2606:4700:10::6816:926 port 443 failed: Resource temporarily unavailable
*   Trying 2606:4700:10::6816:826:443...
* Connect socket 6 over QUIC to 2606:4700:10::6816:826:443
* connect to 2606:4700:10::6816:826 port 443 failed: Resource temporarily unavailable
*   Trying 2606:4700:10::ac43:9eb:443...
* Connect socket 7 over QUIC to 2606:4700:10::ac43:9eb:443
* connect to 2606:4700:10::ac43:9eb port 443 failed: Resource temporarily unavailable
*   Trying 172.67.9.235:443...
* Connect socket 8 over QUIC to 172.67.9.235:443
* connect to 172.67.9.235 port 443 failed: Resource temporarily unavailable
*   Trying 104.22.8.38:443...
* Connect socket 9 over QUIC to 104.22.8.38:443
* connect to 104.22.8.38 port 443 failed: Resource temporarily unavailable
*   Trying 104.22.9.38:443...
* Connect socket 10 over QUIC to 104.22.9.38:443
* connect to 104.22.9.38 port 443 failed: Resource temporarily unavailable
* Failed to connect to cloudflare-quic.com port 443: Resource temporarily unavailable
* Closing connection 0
curl: (7) Failed to connect to cloudflare-quic.com port 443: Resource temporarily unavailable

curl version: 7.76.1-DEV (commit: 3be5ebf)

# curl-http3 -V
curl 7.76.1-DEV (x86_64-pc-linux-gnu) libcurl/7.76.1-DEV GnuTLS/3.7.1 zlib/1.2.11 brotli/1.0.9 zstd/1.4.9 c-ares/1.17.1 libidn2/2.3.0 libssh2/1.9.0 nghttp2/1.43.0 ngtcp2/0.1.0-DEV nghttp3/0.1.0-DEV
Release-Date: XXX
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp 
Features: alt-svc AsynchDNS brotli HTTP2 HTTP3 HTTPS-proxy IDN IPv6 Largefile libz NTLM NTLM_WB SSL TLS-SRP UnixSockets zstd

But the old version works.

# curl-http3 -vI --http3 https://cloudflare-quic.com
*   Trying 2606:4700:10::6816:826:443...
* Connect socket 5 over QUIC to 2606:4700:10::6816:826:443
*   Trying 104.22.8.38:443...
* Connect socket 6 over QUIC to 104.22.8.38:443
* QUIC handshake is completed
* Connected to cloudflare-quic.com () port 443 (#0)
* Using HTTP/3 Stream ID: 0 (easy handle 0xadbb50)
> HEAD / HTTP/3
> Host: cloudflare-quic.com
> user-agent: curl/7.74.0-DEV
> accept: */*
> 
* ngh3_stream_recv returns 0 bytes and EAGAIN
* ngh3_stream_recv returns 0 bytes and EAGAIN
< HTTP/3 200 
HTTP/3 200 
...
* Connection #0 to host cloudflare-quic.com left intact

@bagder
Copy link
Member

@bagder bagder commented Apr 13, 2021

@tatsuhiro-t any chance you can throw a quick eye on this? It looks like one of our more recent adjustments to the changing ngtcp2 APIs broke functionality and you might have a chance to spot that much faster than anyone else...

Loading

@bagder bagder changed the title http3 connect failed: Resource temporarily unavailable http3 connect failed: Resource temporarily unavailable (with ngtcp2) Apr 13, 2021
@tatsuhiro-t
Copy link
Contributor

@tatsuhiro-t tatsuhiro-t commented Apr 13, 2021

There are couple of issues, some are client side, others are, well, version negotiation failure.

Curl uses ALPN h3

curl/lib/vquic/ngtcp2.c

Lines 354 to 355 in 0409c12

alpn = (const uint8_t *)NGHTTP3_ALPN_H3;
alpnlen = sizeof(NGHTTP3_ALPN_H3) - 1;

and at the same time it uses version 0xff00001d

curl/lib/vquic/ngtcp2.c

Lines 816 to 818 in 0409c12

rc = ngtcp2_conn_client_new(&qs->qconn, &qs->dcid, &qs->scid, &path,
NGTCP2_PROTO_VER_MIN, &ng_callbacks,
&qs->settings, &qs->transport_params, NULL, qs);

In my opinion, we should use version 0x00000001 for ALPN h3.

If we make this change, it can get from ngtcp2 server, but it fails in curl (56), which is probably another issue.

If we change version to 0x00000001, it still fails with cloudflare-quc.com because it does not support QUIC v1 yet.
In order to workaround this, use QUIC version 0xff00001d (as is) and ALPN h3-29 (change required). Some servers are still supporting h3-29, so this might work.

Loading

@tatsuhiro-t
Copy link
Contributor

@tatsuhiro-t tatsuhiro-t commented Apr 13, 2021

If we make this change, it can get from ngtcp2 server, but it fails in curl (56), which is probably another issue.

This is ngtcp2 server bug.

Loading

@icebluey
Copy link
Author

@icebluey icebluey commented Apr 14, 2021

@bagder , @tatsuhiro-t
#6886 this fix doesn't seem to work.
Curl v7.76.1
GnuTLS v3.7.1
nghttp3 (commit: 5cccfe32585738b86b3dd8341509273353ae4ca6)
ngtcp2 ( commit: 169c68127b78ea906c96b49b9e18d4f805ab8eda)
Still failed to connect to cloudflare-quic.com port 443: Resource temporarily unavailable.
Failed to connect to www.cloudflare.com port 443: Resource temporarily unavailable.
Failed to connect to www.google.com port 443: Resource temporarily unavailable.

Loading

@bagder
Copy link
Member

@bagder bagder commented Apr 14, 2021

Works for me:

$ ./src/curl -V
curl 7.76.1-DEV (x86_64-pc-linux-gnu) libcurl/7.76.1-DEV OpenSSL/1.1.1g zlib/1.2.11 brotli/1.0.9 zstd/1.4.8 c-ares/1.17.1 libidn2/2.3.0 libpsl/0.21.0 (+libidn2/2.3.0) libssh2/1.9.0_DEV nghttp2/1.42.0-DEV ngtcp2/0.1.0-DEV nghttp3/0.1.0-DEV librtmp/2.3 libgsasl/1.10.0

Loading

@icebluey
Copy link
Author

@icebluey icebluey commented Apr 14, 2021

Works for me:

$ ./src/curl -V
curl 7.76.1-DEV (x86_64-pc-linux-gnu) libcurl/7.76.1-DEV OpenSSL/1.1.1g zlib/1.2.11 brotli/1.0.9 zstd/1.4.8 c-ares/1.17.1 libidn2/2.3.0 libpsl/0.21.0 (+libidn2/2.3.0) libssh2/1.9.0_DEV nghttp2/1.42.0-DEV ngtcp2/0.1.0-DEV nghttp3/0.1.0-DEV librtmp/2.3 libgsasl/1.10.0

Your situation is with OpenSSL/1.1.1g?

I built ngtcp2 with gnutls.

# curl-http3 -V
curl 7.76.1 (x86_64-pc-linux-gnu) libcurl/7.76.1 GnuTLS/3.7.1 zlib/1.2.11 brotli/1.0.9 zstd/1.4.9 c-ares/1.17.1 libidn2/2.3.0 libssh2/1.9.0 nghttp2/1.43.0 ngtcp2/0.1.0-DEV nghttp3/0.1.0-DEV
Release-Date: 2021-04-14
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp 
Features: alt-svc AsynchDNS brotli HTTP2 HTTP3 HTTPS-proxy IDN IPv6 Largefile libz NTLM NTLM_WB SSL TLS-SRP UnixSockets zstd

Loading

@tatsuhiro-t
Copy link
Contributor

@tatsuhiro-t tatsuhiro-t commented Apr 14, 2021

It looks like boringssl backed server does not like ClientHello generated by GnuTLS. It is hard for me to debug this because it is TLS internal stuff.

Loading

@bagder
Copy link
Member

@bagder bagder commented Apr 14, 2021

@icebluey before we landed #6886, this problem existed for ngtcp2 using either openssl or gnutls and I verified the fix with my openssl build, yes.

Loading

@D4v1dH03
Copy link
Contributor

@D4v1dH03 D4v1dH03 commented Apr 15, 2021

"Resource temporarily unavailable" still happens to me with GnuTLS build from GitLab as of GnuTLS commit a6a45ad0

Loading

@curl curl locked and limited conversation to collaborators Apr 15, 2021
@bagder
Copy link
Member

@bagder bagder commented Apr 15, 2021

Comments on closed issues mostly get lost/ignored. I filed a new issue for this issue for the gnutls flavor.

Loading

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

5 participants