Why does not work quic ?

[zino7825]

Detailed Description of the Problem

chrome browser has been connected to server with h2 protocol after quic setting!.

Expected Behavior

use h3 proto in chrome.

Steps to Reproduce the Behavior

1. install quictls 3.0.3+quic
2. install haproxy 2.6.1 with USE_QUIC=1
3. set config ( bind quic4@:443 crt-list xxx and set response header alt-svc)

Do you have any idea what may have caused this?

I have no idea.
Let me know what is my fault?

Do you have an idea how to solve the issue?

No response

What is your configuration?

bind :443 tfo ssl curves X25519:P-256 crt xxx alpn h2
bind quic4@:443 tfo ssl curves X25519:P-256 crt xxx alpn h3

http-response set-header alt-svc "h3=\":443\";ma=900;"

use_backend %[hdr(host),map(/etc/haproxy/host.map,default)]

Output of haproxy -vv

HAProxy version 2.6.1 2022/06/21 - https://haproxy.org/
Status: long-term supported branch - will stop receiving fixes around Q2 2027.
Known bugs: http://www.haproxy.org/bugs/bugs-2.6.1.html
Running on: Linux 5.11.10-1.el7.elrepo.x86_64 #1 SMP Thu Mar 25 09:56:28 EDT 2021 x86_64
Build options :
  TARGET  = linux-glibc
  CPU     = native
  CC      = cc
  CFLAGS  = -O2 -march=native -g -Wall -Wextra -Wundef -Wdeclaration-after-statement -Wfatal-errors -Wtype-limits -fwrapv -Wno-address-of-packed-member -Wno-unused-label -Wno-sign-compare -Wno-unused-parameter -Wno-clobbered -Wno-missing-field-initializers -Wno-cast-function-type -Wno-string-plus-int -Wno-atomic-alignment -DHAPROXY_TARGET_VERSION=260 -DTLS_TICKETS_NO=4
  OPTIONS = USE_PCRE=1 USE_PCRE_JIT=1 USE_OPENSSL=1 USE_LUA=1 USE_ZLIB=1 USE_QUIC=1
  DEBUG   = -DDEBUG_STRICT -DDEBUG_MEMORY_POOLS

Feature list : +EPOLL -KQUEUE +NETFILTER +PCRE +PCRE_JIT -PCRE2 -PCRE2_JIT +POLL +THREAD +BACKTRACE -STATIC_PCRE -STATIC_PCRE2 +TPROXY +LINUX_TPROXY +LINUX_SPLICE +LIBCRYPT +CRYPT_H -ENGINE +GETADDRINFO +OPENSSL +LUA +ACCEPT4 -CLOSEFROM +ZLIB -SLZ +CPU_AFFINITY +TFO +NS +DL +RT -DEVICEATLAS -51DEGREES -WURFL -SYSTEMD -OBSOLETE_LINKER +PRCTL -PROCCTL +THREAD_DUMP -EVPORTS -OT +QUIC -PROMEX -MEMORY_PROFILING

Default settings :
  bufsize = 16384, maxrewrite = 1024, maxpollevents = 200

Built with multi-threading support (MAX_THREADS=64, default=20).
Built with OpenSSL version : OpenSSL 3.0.3+quic 3 May 2022
Running on OpenSSL version : OpenSSL 3.0.3+quic 3 May 2022

Last Outputs and Backtraces

No response

Additional Information

No response

[a-denoyelle]

It's difficult to understand fully the behavior of browsers with QUIC. In my experience, chromium was more restrictive, as it seems to only accept certificates signed by an authority recognized by the browser. With firefox this restriction was not present.

As QUIC is a still recent protocol, try to use the more recent version of chrome/firefox. If you can, try to record a network capture to see if there is some UDP traffic as sometimes debugger tools of the browsers does not report QUIC connections.

Also, if you want to first test your QUIC frontend, you can try to use a command-line tool with QUIC support to directly reach the UPD endpoint. For example, curl has experimental support of QUIC but this requires to compile it by hand.

[wtarreau]

For me, chromium starts with H2 and only switches to H3 on the second reload (i.e. press Ctrl-R twice). There are probably a number of heuristics in place to avoid the impact on the vast majority of non-QUIC traffic, that require to perform background probes or certain such things in order to decide whether to switch to H3 for subsequent requests or not.

[wtarreau]

By the way, for me FF91 stopped using QUIC on haproxy.org a week or two ago, and there was no way to make it accept it again, it would only exchange a few packets and switch back to H2. And today without any explanation, it's back to work; the process was not even stopped! I think browsers perform some negative caching of issues in order not to try again after a site showed some issues.

[zino7825]

It's difficult to understand fully the behavior of browsers with QUIC. In my experience, chromium was more restrictive, as it seems to only accept certificates signed by an authority recognized by the browser. With firefox this restriction was not present.

As QUIC is a still recent protocol, try to use the more recent version of chrome/firefox. If you can, try to record a network capture to see if there is some UDP traffic as sometimes debugger tools of the browsers does not report QUIC connections.

Also, if you want to first test your QUIC frontend, you can try to use a command-line tool with QUIC support to directly reach the UPD endpoint. For example, curl has experimental support of QUIC but this requires to compile it by hand.

but I can access google.com with http3 on current browser.

[wtarreau]

but I can access google.com with http3 on current browser.

Just like mine used to work on quic.cloudflare.com while not on haproxy.org, and recently started to refuse to use H3 on both quic.cloudflare.com and haproxy.org for a few days, then decided to accept it again on both without even restarting it. I really don't know how they're deciding to stop using H3/QUIC on a site, but there's definitely some caching and we don't know where to search for diagnostic info.

[zino7825]

but I can access google.com with http3 on current browser.

Just like mine used to work on quic.cloudflare.com while not on haproxy.org, and recently started to refuse to use H3 on both quic.cloudflare.com and haproxy.org for a few days, then decided to accept it again on both without even restarting it. I really don't know how they're deciding to stop using H3/QUIC on a site, but there's definitely some caching and we don't know where to search for diagnostic info.

You mean, is this problem related to the Chrome browser?

[wtarreau]

I'm saying that it possibly isn't even a problem and might be the expected behavior as a consequence of some initially failed tests. We've all gone through this during debugging and that's really painful. What is really annoying at the moment with browsers is that they provide absolutely zero information to debug such situations. So it's not even possible to say whether it's expected or not.

As Amaury said, you should really try a second tool, and check with network traces or by enabling QUIC traces in haproxy whether the browser and haproxy engage in a conversation or not.

Also I'm seeing that you're having "quic4@:443". In case there are multiple addresses on your machine (I don't know), this can cause problems as UDP is connection-less, so it's possible that the browser speaks to haproxy on one IP address and that responses are sent from another one chosen by the operating system. Just in case that would be related, please adjust your config to bind to an explicit address, e.g. "quic4@192.168.1.1:443".

[zino7825]

@wtarreau I got it.
I tried to set quic-force when starting chrome ( chrome --enable-quic --origin-to-force-quic-on=xxx.com:443 )
and I checked that h3 protocol was succeed.
it's really weird things. -_-;

I will also try to set the dedecated IP. thx!

[wtarreau]

Interesting and very good to know for our debugging sessions, thanks very much for the hint!
So that proves that your connectivity is OK even if we still don't know why by default it doesn't want to use it, but it's making progress.

[wtarreau]

Many similar issues have been fixed over the last year and the experience got much better overall. I suggest to close this one and open a new one with new info if similar problems arise with an up-to-date version now.