nginx 1.23.1

[macbre]

http://nginx.org/en/CHANGES

[macbre]

2022-04-12T19:33:51.3448318Z * connect to 127.0.0.1 port 8889 failed: Connection timed out
2022-04-12T19:33:51.3448786Z * Failed to connect to localhost port 8889: Connection timed out
2022-04-12T19:33:51.3449086Z 
2022-04-12T19:33:51.3449449Z   0     0    0     0    0     0      0      0 --:--:--  0:02:37 --:--:--     0
2022-04-12T19:33:51.3450075Z * Closing connection 0
2022-04-12T19:33:51.3450610Z curl: (28) quiche: recv() unexpectedly returned -1 (errno: 111, socket 5)

[macbre]

#15 52.55 /usr/src/headers-more-nginx-module-0.33/src/ngx_http_headers_more_headers_in.c:162:50: error: 'ngx_http_headers_in_t' has no member named 'cookies'; did you mean 'cookie'?

openresty/headers-more-nginx-module#132

[macbre]

#15 52.22 /ngx_http_geoip2_module/ngx_http_geoip2_module.c:175:58: error: passing argument 3 of 'ngx_http_get_forwarded_addr' from incompatible pointer type [-Werror=incompatible-pointer-types]
#15 52.22   175 |             (void) ngx_http_get_forwarded_addr(r, &addr, xfwd, NULL,
#15 52.22       |                                                          ^~~~
#15 52.22       |                                                          |
#15 52.22       |                                                          ngx_array_t *
#15 52.22 In file included from src/http/ngx_http.h:38,
#15 52.22                  from /ngx_http_geoip2_module/ngx_http_geoip2_module.c:10:
#15 52.22 src/http/ngx_http_core_module.h:536:22: note: expected 'ngx_table_elt_t *' {aka 'struct ngx_table_elt_s *'} but argument is of type 'ngx_array_t *'
#15 52.22   536 |     ngx_table_elt_t *headers, ngx_str_t *value, ngx_array_t *proxies,
#15 52.22       |     ~~~~~~~~~~~~~~~~~^~~~~~~

[macbre]

2022-06-27T14:10:50.0626314Z * Connect socket 5 over QUIC to ::1:8889
2022-06-27T14:10:50.0630070Z * Sent QUIC client Initial, ALPN: h3-29,h3-28,h3-27
2022-06-27T14:10:50.0631603Z * quiche: recv() unexpectedly returned -1 (errno: 111, socket 5)
2022-06-27T14:10:50.0632526Z * connect to ::1 port 8889 failed: Connection refused
2022-06-27T14:10:50.0633133Z *   Trying 127.0.0.1:8889...
2022-06-27T14:10:50.0633516Z * Connect socket 6 over QUIC to 127.0.0.1:8889
2022-06-27T14:10:50.0634306Z * Sent QUIC client Initial, ALPN: h3-29,h3-28,h3-27
2022-06-27T14:10:50.0635096Z * quiche_conn_recv() == -3
2022-06-27T14:10:50.0635554Z *   Trying 127.0.0.1:8889...

And then it finally times out:

2022-06-27T14:13:26.6662296Z   0     0    0     0    0     0      0      0 --:--:--  0:02:35 --:--:--     0* After 150000ms connect time, move on!
2022-06-27T14:13:26.6662978Z * connect to 127.0.0.1 port 8889 failed: Connection timed out
2022-06-27T14:13:26.6663641Z * Failed to connect to localhost port 8889: Connection timed out
2022-06-27T14:13:26.6663964Z 
2022-06-27T14:13:26.6664363Z   0     0    0     0    0     0      0      0 --:--:--  0:02:36 --:--:--     0
2022-06-27T14:13:26.6664724Z * Closing connection 0
2022-06-27T14:13:26.6740472Z curl: (28) quiche: recv() unexpectedly returned -1 (errno: 111, socket 5)
2022-06-27T14:13:26.7327797Z + grep --fixed-strings '< HTTP/3 200' /tmp/h3
2022-06-27T14:13:31.7339785Z ##[error]Process completed with exit code 1.

[macbre]

cloudflare/quiche#91

[macbre]

*   Trying 127.0.0.1:8889...
* Connect socket 14079 over QUIC to 127.0.0.1:8889
* Sent QUIC client Initial, ALPN: h3-29,h3-28,h3-27
* quiche_conn_recv() == -3

https://github.com/cloudflare/quiche/blob/06973ef54834cc00f24fde44812529b35821455d/quiche/include/quiche.h#L74-L75 :

// The provided packet cannot be parsed because its version is unknown.
QUICHE_ERR_UNKNOWN_VERSION = -3

The latest nginx container responds with:

* Connect socket 5 over QUIC to ::1:8889
* Sent QUIC client Initial, ALPN: h3-29,h3-28,h3-27
* quiche: recv() unexpectedly returned -1 (errno: 111, socket 5)
* connect to ::1 port 8889 failed: Connection refused
*   Trying 127.0.0.1:8889...
* Connect socket 6 over QUIC to 127.0.0.1:8889
* Sent QUIC client Initial, ALPN: h3-29,h3-28,h3-27
* Connected to localhost () port 8889 (#0)
* h3 [:method: GET]
* h3 [:path: /]
* h3 [:scheme: https]
* h3 [:authority: localhost:8889]
* h3 [user-agent: curl/7.76.1-DEV]
* h3 [accept: */*]
* Using HTTP/3 Stream ID: 0 (easy handle 0x55f768e191b0)
> GET / HTTP/3
> Host: localhost:8889
> user-agent: curl/7.76.1-DEV
> accept: */*

[imraan-go]

Is there any problem with version 1.23.1?

[macbre]

Yes, the http3 connections do not pass curl tests.

[imraan-go]

Strangely browsers such as chrome,safari seem to work just fine. Tried several browsers and online testing tools, all works OK.

[macbre]

Interesting, however we do need a CI check for http3 connectivity. Thanks for checking!

[imraan-go]

After extensive research, I figured that the curl that you are using is old version which only accepts h3-29,h3-28 and h3-27. This draft versions has been removed from nginx. See here
https://hg.nginx.org/nginx-quic/rev/ca78312db071
https://hg.nginx.org/nginx-quic/rev/d8865baab732

Currently nginx version only support QUIC V1. So we just have to use updated curl binary thats built with quiche with QUIC V1.

[macbre]

Thanks for your research, @imraan-go !

That was my wild guess as well. So now we need to have a custom curl build to test our custom nginx build. Fun 😆

[macbre]

@imraan-go - my own curl build (with quiche and boringssl) is almost there. And it does work fine with this nginx container 🎉

$ docker run --rm --network=host macbre/curl3 curl -sIL --insecure https://0.0.0.0:8889 --http3 | grep -i http
HTTP/3 200

And curl says (h3 proto is added here compared to the version we previously were using):

* Sent QUIC client Initial, ALPN: h3,h3-29,h3-28,h3-27

[macbre]

So, off we go 🚀

[imraan-go]

Great 🥳 . I've already been using this version in production without any issues so far.