Experimental HTTP/3 support

[buchdag]

Contrary to what I thought, the container version of nginx 1.25.x is built with --with-http_v3_module:

nduchon@host:~$ docker run --rm nginx:1.25.1 nginx -V
nginx version: nginx/1.25.1
built by gcc 12.2.0 (Debian 12.2.0-14) 
built with OpenSSL 3.0.9 30 May 2023
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-http_v3_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-g -O2 -ffile-prefix-map=/data/builder/debuild/nginx-1.25.1/debian/debuild-base/nginx-1.25.1=. -fstack-protector-strong -Wformat -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fPIC' --with-ld-opt='-Wl,-z,relro -Wl,-z,now -Wl,--as-needed -pie'

This opens up the possibility of experimental HTTP/3 support in nginx-proxy.

The documentation is not super clear at the moment. There is an http3 directive that supposedly Enables HTTP/3 protocol negotiation, but it's not present on any example configuration provided by nginx and there is no real explanation of what it does.

I've played a bit with it and still fails to get a truly working minimal POC. I end up generating a config that look like this:

server {
    server_name _; # This is just an invalid value which will never trigger on a real hostname.
    server_tokens off;
    http2 on;
    listen 80;
    listen 443 ssl;
    listen 443 quic reuseport;
    ssl_session_cache shared:SSL:50m;
    ssl_session_tickets off;
    access_log /var/log/nginx/access.log vhost;
    # No default.crt certificate found for this vhost, so force nginx to emit a
    # TLS error if the client connects via https.
    ssl_ciphers aNULL;
    set $empty "";
    ssl_certificate data:$empty;
    ssl_certificate_key data:$empty;
    if ($https) {
        return 444;
    }
    return 503;
}
# [www.foo.bar/](http://www.foo.bar/)
upstream [www.foo.bar](http://www.foo.bar/) {
    # Container: nginx-proxy-test1-1
    #     networks:
    #         nginx-proxy_default (reachable)
    #     IP address: 192.168.224.2
    #     exposed ports: 80/tcp
    #     default port: 80
    #     using port: 80
    server 192.168.224.2:80;
}
server {
    server_name www.foo.bar;
    listen 80;
    access_log /var/log/nginx/access.log vhost;
    # Do not HTTPS redirect Let's Encrypt ACME challenge
    location ^~ /.well-known/acme-challenge/ {
        auth_basic off;
        auth_request off;
        allow all;
        root /usr/share/nginx/html;
        try_files $uri =404;
        break;
    }
    location / {
        return 301 https://$host$request_uri;
    }
}
server {
    server_name www.foo.bar;
    http2 on;
    access_log /var/log/nginx/access.log vhost;
    listen 443 ssl;
    listen 443 quic;
    ssl_session_timeout 5m;
    ssl_session_cache shared:SSL:50m;
    ssl_session_tickets off;
    ssl_certificate /etc/nginx/certs/www.foo.bar.crt;
    ssl_certificate_key /etc/nginx/certs/www.foo.bar.key;
    set $sts_header "";
    if ($https) {
        set $sts_header "max-age=31536000";
    }
    add_header Strict-Transport-Security $sts_header always;
    location / {
        add_header alt-svc 'h3=":443"; ma=86400;';
        proxy_pass http://www.foo.bar;
        set $upstream_keepalive false;
    }
}

Testing with curl (built with HTTP/3 support), the connexion appears to work, but the proxied nginx containers answer with an HTTP 400 error:

nduchon@host:~$ /usr/local/bin/curl -v -k --http3 https://www.foo.bar/
* processing: https://www.foo.bar/
*   Trying 127.0.0.1:443...
* Skipped certificate verification
* Connected to [www.foo.bar](http://www.foo.bar/) (127.0.0.1) port 443
* using HTTP/3
* Using HTTP/3 Stream ID: 0
> GET / HTTP/3
> Host: [www.foo.bar](http://www.foo.bar/)
> User-Agent: curl/8.2.0-DEV
> Accept: */*
> 
< HTTP/3 400 
< server: nginx/1.25.1
< date: Sun, 09 Jul 2023 17:33:17 GMT
< content-type: text/html
< content-length: 157
< 
<html>
<head><title>400 Bad Request</title></head>
<body>
<center><h1>400 Bad Request</h1></center>
<hr><center>nginx/1.25.1</center>
</body>
</html>
* Connection #0 to host [www.foo.bar](http://www.foo.bar/) left intact

Testing the exact same request with --http2 works fine

nduchon@host:~$  /usr/local/bin/curl -v -k --http2 https://www.foo.bar/
* processing: https://www.foo.bar/
*   Trying 127.0.0.1:443...
* Connected to [www.foo.bar](http://www.foo.bar/) (127.0.0.1) port 443
* ALPN: offers h2,http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN: server accepted h2
* Server certificate:
*  subject: C=FR; ST=Paris; L=Paris; O=Duch; [CN=www.foo.bar](http://cn%3Dwww.foo.bar/)
*  start date: Jul  5 17:16:37 2023 GMT
*  expire date: Jul  4 17:16:37 2024 GMT
*  issuer: C=FR; ST=Paris; L=Paris; O=Duch; [CN=www.foo.bar](http://cn%3Dwww.foo.bar/)
*  SSL certificate verify result: self-signed certificate (18), continuing anyway.
* using HTTP/2
* h2 [:method: GET]
* h2 [:scheme: https]
* h2 [:authority: www.foo.bar]
* h2 [:path: /]
* h2 [user-agent: curl/8.2.0-DEV]
* h2 [accept: */*]
* Using Stream ID: 1
> GET / HTTP/2
> Host: [www.foo.bar](http://www.foo.bar/)
> User-Agent: curl/8.2.0-DEV
> Accept: */*
> 
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
< HTTP/2 200 
< server: nginx/1.25.1
< date: Sun, 09 Jul 2023 17:33:31 GMT
< content-type: text/html
< content-length: 615
< last-modified: Tue, 13 Jun 2023 17:34:28 GMT
< etag: "6488a8a4-267"
< accept-ranges: bytes
< alt-svc: h3=":443"; ma=86400;
< 
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
* Connection #0 to host [www.foo.bar](http://www.foo.bar/) left intact

[Knapoc]

Actually I got http3 support working by modifying the template.

Generated config

server {
    server_name _; # This is just an invalid value which will never trigger on a real hostname.
    server_tokens off;
    http2 on;
    http3 on;
    listen 8080;
    listen 10443 ssl;
    listen 10443 quic reuseport;
    ssl_session_cache shared:SSL:50m;
    ssl_session_tickets off;
    access_log /var/log/nginx/access.log vhost;
    ssl_certificate /etc/nginx/certs/default.crt;
    ssl_certificate_key /etc/nginx/certs/default.key;
    return 503;
}
# foo.bar.com/
upstream foo.bar.com {
    # Container: authentik-server
    #     networks:
    #         proxy (reachable)
    #         webapps_ak_net (unreachable)
    #         webapps_cache_net (unreachable)
    #         webapps_postgre_net (unreachable)
    #     IP address: 172.21.0.6
    #     exposed ports: (none)
    #     default port: 80
    #     using port: 9000
    server 172.21.0.6:9000;
}
server {
    server_name foo.bar.com;
    listen 8080 ;
    access_log /var/log/nginx/access.log vhost;
    # Do not HTTPS redirect Let's Encrypt ACME challenge
    location ^~ /.well-known/acme-challenge/ {
        auth_basic off;
        auth_request off;
        allow all;
        root /usr/share/nginx/html;
        try_files $uri =404;
        break;
    }
    location / {
        return 301 https://$host:10443$request_uri;
    }
}
server {
    server_name foo.bar.com;
    http2 on;
    http3 on;
    access_log /var/log/nginx/access.log vhost;
    listen 10443 ssl ;
    listen 10443 quic ;
    ssl_session_timeout 5m;
    ssl_session_cache shared:SSL:50m;
    ssl_session_tickets off;
    ssl_certificate /etc/nginx/certs/foo.bar.com.crt;
    ssl_certificate_key /etc/nginx/certs/foo.bar.com.key;
    ssl_dhparam /etc/nginx/certs/foo.bar.com.dhparam.pem;
    ssl_stapling on;
    ssl_stapling_verify on;
    ssl_trusted_certificate /etc/nginx/certs/foo.bar.com.chain.pem;
    set $sts_header "";
    if ($https) {
        set $sts_header "max-age=31536000";
    }
    add_header Strict-Transport-Security $sts_header always;
    include /etc/nginx/vhost.d/default;
    location / {
        proxy_pass http://foo.bar.com;
        set $upstream_keepalive false;
        add_header Alt-Svc 'h3=":443"; ma=86400';
    }
}

curl output using http3

knapoc: ~$ curl -v -k --http3 https://foo.bar.com
*   Trying xxx.xxx.xxx.xxx:443...
* found 135 certificates in /opt/local/share/curl/curl-ca-bundle.crt
* GnuTLS ciphers: NORMAL:-ARCFOUR-128:-CTYPE-ALL:+CTYPE-X509:-VERS-SSL3.0
* Skipped certificate verification
* Connected to foo.bar.com (84.113.128.54) port 443 (#0)
* using HTTP/3
* Using HTTP/3 Stream ID: 0 (easy handle 0x14c80fa00)
> GET / HTTP/3
> Host: foo.bar.com
> User-Agent: curl/8.1.2
> Accept: */*
> 
< HTTP/3 302 
< server: nginx
< date: Thu, 13 Jul 2023 07:47:53 GMT
< content-type: text/html; charset=utf-8
< content-length: 0
< location: /
< referrer-policy: same-origin
< vary: Accept-Encoding
< vary: Cookie
< x-content-type-options: nosniff
< x-frame-options: DENY
< alt-svc: h3=":443"; ma=86400
< 
* Connection #0 to host foo.bar.com left intact

curl output using http2

knapoc: ~$ curl -v -k --http2 https://foo.bar.com
*   Trying xxx.xxx.xxx.xxx:443...
* Connected to foo.bar.com (84.113.128.54) port 443 (#0)
* found 135 certificates in /opt/local/share/curl/curl-ca-bundle.crt
* GnuTLS ciphers: NORMAL:-ARCFOUR-128:-CTYPE-ALL:+CTYPE-X509:-VERS-SSL3.0
* ALPN: offers h2,http/1.1
* SSL connection using TLS1.3 / ECDHE_RSA_AES_256_GCM_SHA384
*   server certificate verification SKIPPED
*   server certificate status verification SKIPPED
*   common name: bar.com (matched)
*   server certificate expiration date OK
*   server certificate activation date OK
*   certificate public key: EC/ECDSA
*   certificate version: #3
*   subject: CN=bar.com
*   start date: Thu, 29 Jun 2023 10:20:07 GMT
*   expire date: Wed, 27 Sep 2023 10:20:06 GMT
*   issuer: C=US,O=Let's Encrypt,CN=R3
* ALPN: server accepted h2
* using HTTP/2
* h2 [:method: GET]
* h2 [:scheme: https]
* h2 [:authority: foo.bar.com]
* h2 [:path: /]
* h2 [user-agent: curl/8.1.2]
* h2 [accept: */*]
* Using Stream ID: 1 (easy handle 0x000000000)
> GET / HTTP/2
> Host: foo.bar.com
> User-Agent: curl/8.1.2
> Accept: */*
> 
< HTTP/2 302 
< server: nginx
< date: Thu, 13 Jul 2023 07:47:59 GMT
< content-type: text/html; charset=utf-8
< content-length: 0
< location: /
< referrer-policy: same-origin
< vary: Accept-Encoding
< vary: Cookie
< x-content-type-options: nosniff
< x-frame-options: DENY
< alt-svc: h3=":443"; ma=86400
< 
* Connection #0 to host foo.bar.com left intact

Template

Note: here is another modification present to enable network separation between a nginx container and a docker-get container when using separate containers

# nginx-proxy{{ if $.Env.NGINX_PROXY_VERSION }} version : {{ $.Env.NGINX_PROXY_VERSION }}{{ end }}

{{- /*
     * Global values.  Values are stored in this map rather than in individual
     * global variables so that the values can be easily passed to embedded
     * templates.  (Go templates cannot access variables outside of their own
     * scope.)
     */}}
{{- $globals := dict }}
{{- $_ := set $globals "containers" $ }}
{{- $_ := set $globals "Env" $.Env }}
{{- $_ := set $globals "Docker" $.Docker }}
{{- $_ := set $globals "CurrentContainer" (where $globals.containers "ID" $globals.Docker.CurrentContainerID | first) }}
{{- $_ := set $globals "NginxContainer" (whereLabelExists $globals.containers "com.github.nginx-proxy.nginx-proxy.nginx-proxy" | first) }}
{{- $_ := set $globals "default_cert_ok" (and (exists "/etc/nginx/certs/default.crt") (exists "/etc/nginx/certs/default.key")) }}
{{- $_ := set $globals "external_http_port" (coalesce $globals.Env.HTTP_PORT "80") }}
{{- $_ := set $globals "external_https_port" (coalesce $globals.Env.HTTPS_PORT "443") }}
{{- $_ := set $globals "sha1_upstream_name" (parseBool (coalesce $globals.Env.SHA1_UPSTREAM_NAME "false")) }}
{{- $_ := set $globals "default_root_response" (coalesce $globals.Env.DEFAULT_ROOT "404") }}
{{- $_ := set $globals "trust_downstream_proxy" (parseBool (coalesce $globals.Env.TRUST_DOWNSTREAM_PROXY "true")) }}
{{- $_ := set $globals "access_log" (or (and (not $globals.Env.DISABLE_ACCESS_LOGS) "access_log /var/log/nginx/access.log vhost;") "") }}
{{- $_ := set $globals "enable_ipv6" (parseBool (coalesce $globals.Env.ENABLE_IPV6 "false")) }}
{{- $_ := set $globals "enable_http3" (parseBool (coalesce $globals.Env.ENABLE_HTTP3 "false")) }}
{{- $_ := set $globals "ssl_policy" (or ($globals.Env.SSL_POLICY) "Mozilla-Intermediate") }}
{{- $_ := set $globals "vhosts" (dict) }}
{{- $_ := set $globals "networks" (dict) }}
# Networks available to the container labeled "com.github.nginx-proxy.nginx-proxy.nginx-proxy" or the one running docker-gen (which are assumed to
# match the networks available to the container running nginx):
{{- /*
     * Note: $globals.CurrentContainer may be nil in some circumstances due to
     * <https://github.com/nginx-proxy/docker-gen/issues/458>.  For more context
     * see <https://github.com/nginx-proxy/nginx-proxy/issues/2189>.
     */}}
{{- if $globals.NginxContainer }}
    {{- range sortObjectsByKeysAsc $globals.NginxContainer.Networks "Name" }}
        {{- $_ := set $globals.networks .Name . }}
#     {{ .Name }}
    {{- else }}
#     (none)
    {{- end }}
{{- else if $globals.CurrentContainer }}
    {{- range sortObjectsByKeysAsc $globals.CurrentContainer.Networks "Name" }}
        {{- $_ := set $globals.networks .Name . }}
#     {{ .Name }}
    {{- else }}
#     (none)
    {{- end }}
{{- else }}
# /!\ WARNING: Failed to find the Docker container running docker-gen.  All
#              upstream (backend) application containers will appear to be
#              unreachable.  Try removing the -only-exposed and -only-published
#              arguments to docker-gen if you pass either of those.  See
#              <https://github.com/nginx-proxy/docker-gen/issues/458>.
{{- end }}

{{- /*
     * Template used as a function to get a container's IP address.  This
     * template only outputs debug comments; the IP address is "returned" by
     * storing the value in the provided dot dict.
     *
     * The provided dot dict is expected to have the following entries:
     *   - "globals": Global values.
     *   - "container": The container's RuntimeContainer struct.
     *
     * The return value will be added to the dot dict with key "ip".
     */}}
{{- define "container_ip" }}
    {{- $ip := "" }}
    #     networks:
    {{- range sortObjectsByKeysAsc $.container.Networks "Name" }}
        {{- /*
             * TODO: Only ignore the "ingress" network for Swarm tasks (in case
             * the user is not using Swarm mode and names a network "ingress").
             */}}
        {{- if eq .Name "ingress" }}
    #         {{ .Name }} (ignored)
            {{- continue }}
        {{- end }}
        {{- if eq .Name "host" }}
            {{- /* Handle containers in host nework mode */}}
            {{- if (index $.globals.networks "host") }}
    #         both container and proxy are in host network mode, using localhost IP
                {{- $ip = "127.0.0.1" }}
                {{- continue }}
            {{- end }}
            {{- range sortObjectsByKeysAsc $.globals.CurrentContainer.Networks "Name" }}
                {{- if and . .Gateway }}
    #         container is in host network mode, using {{ .Name }} gateway IP
                    {{- $ip = .Gateway }}
                    {{- break }}
                {{- end }}
            {{- end }}
            {{- if $ip }}
                {{- continue }}
            {{- end }}
        {{- end }}
        {{- if and (not (index $.globals.networks .Name)) (not $.globals.networks.host) }}
    #         {{ .Name }} (unreachable)
            {{- continue }}
        {{- end }}
        {{- /*
             * Do not emit multiple `server` directives for this container if it
             * is reachable over multiple networks.  This avoids accidentally
             * inflating the effective round-robin weight of a server due to the
             * redundant upstream addresses that nginx sees as belonging to
             * distinct servers.
             */}}
        {{- if $ip }}
    #         {{ .Name }} (ignored; reachable but redundant)
            {{- continue }}
        {{- end }}
    #         {{ .Name }} (reachable)
        {{- if and . .IP }}
            {{- $ip = .IP }}
        {{- else }}
    #             /!\ No IP for this network!
        {{- end }}
    {{- else }}
    #         (none)
    {{- end }}
    #     IP address: {{ if $ip }}{{ $ip }}{{ else }}(none usable){{ end }}
    {{- $_ := set $ "ip" $ip }}
{{- end }}

{{- /*
     * Template used as a function to get the port of the server in the given
     * container.  This template only outputs debug comments; the port is
     * "returned" by storing the value in the provided dot dict.
     *
     * The provided dot dict is expected to have the following entries:
     *   - "container": The container's RuntimeContainer struct.
     *
     * The return value will be added to the dot dict with key "port".
     */}}
{{- define "container_port" }}
    {{- /* If only 1 port exposed, use that as a default, else 80. */}}
    #     exposed ports:{{ range sortObjectsByKeysAsc $.container.Addresses "Port" }} {{ .Port }}/{{ .Proto }}{{ else }} (none){{ end }}
    {{- $default_port := when (eq (len $.container.Addresses) 1) (first $.container.Addresses).Port "80" }}
    #     default port: {{ $default_port }}
    {{- $port := or $.container.Env.VIRTUAL_PORT $default_port }}
    #     using port: {{ $port }}
    {{- $addr_obj := where $.container.Addresses "Port" $port | first }}
    {{- if and $addr_obj $addr_obj.HostPort }}
    #         /!\ WARNING: Virtual port published on host.  Clients
    #                      might be able to bypass nginx-proxy and
    #                      access the container's server directly.
    {{- end }}
    {{- $_ := set $ "port" $port }}
{{- end }}

{{- define "ssl_policy" }}
    {{- if eq .ssl_policy "Mozilla-Modern" }}
    ssl_protocols TLSv1.3;
        {{- /*
             * nginx currently lacks ability to choose ciphers in TLS 1.3 in
             * configuration; see https://trac.nginx.org/nginx/ticket/1529.  A
             * possible workaround can be modify /etc/ssl/openssl.cnf to change
             * it globally (see
             * https://trac.nginx.org/nginx/ticket/1529#comment:12).  Explicitly
             * set ngnix default value in order to allow single servers to
             * override the global http value.
             */}}
    ssl_ciphers HIGH:!aNULL:!MD5;
    ssl_prefer_server_ciphers off;
    {{- else if eq .ssl_policy "Mozilla-Intermediate" }}
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers 'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-CHACHA20-POLY1305';
    ssl_prefer_server_ciphers off;
    {{- else if eq .ssl_policy "Mozilla-Old" }}
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
    ssl_ciphers 'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:DHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES256-SHA256:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA';
    ssl_prefer_server_ciphers on;
    {{- else if eq .ssl_policy "AWS-TLS-1-2-2017-01" }}
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers 'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:AES128-GCM-SHA256:AES128-SHA256:AES256-GCM-SHA384:AES256-SHA256';
    ssl_prefer_server_ciphers on;
    {{- else if eq .ssl_policy "AWS-TLS-1-1-2017-01" }}
    ssl_protocols TLSv1.1 TLSv1.2 TLSv1.3;
    ssl_ciphers 'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:AES128-GCM-SHA256:AES128-SHA256:AES128-SHA:AES256-GCM-SHA384:AES256-SHA256:AES256-SHA';
    ssl_prefer_server_ciphers on;
    {{- else if eq .ssl_policy "AWS-2016-08" }}
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
    ssl_ciphers 'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:AES128-GCM-SHA256:AES128-SHA256:AES128-SHA:AES256-GCM-SHA384:AES256-SHA256:AES256-SHA';
    ssl_prefer_server_ciphers on;
    {{- else if eq .ssl_policy "AWS-2015-05" }}
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
    ssl_ciphers 'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:AES128-GCM-SHA256:AES128-SHA256:AES128-SHA:AES256-GCM-SHA384:AES256-SHA256:AES256-SHA:DES-CBC3-SHA';
    ssl_prefer_server_ciphers on;
    {{- else if eq .ssl_policy "AWS-2015-03" }}
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
    ssl_ciphers 'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:AES128-GCM-SHA256:AES128-SHA256:AES128-SHA:AES256-GCM-SHA384:AES256-SHA256:AES256-SHA:DHE-DSS-AES128-SHA:DES-CBC3-SHA';
    ssl_prefer_server_ciphers on;
    {{- else if eq .ssl_policy "AWS-2015-02" }}
    ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
    ssl_ciphers 'ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES128-SHA:DHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:AES128-GCM-SHA256:AES128-SHA256:AES128-SHA:AES256-GCM-SHA384:AES256-SHA256:AES256-SHA:DHE-DSS-AES128-SHA';
    ssl_prefer_server_ciphers on;
    {{- end }}
{{- end }}

{{- define "location" }}
    {{- $override := printf "/etc/nginx/vhost.d/%s_%s_location_override" .Host (sha1 .Path) }}
    {{- if and (eq .Path "/") (not (exists $override)) }}
        {{- $override = printf "/etc/nginx/vhost.d/%s_location_override" .Host }}
    {{- end }}
    {{- if exists $override }}
    include {{ $override }};
    {{- else }}
        {{- $keepalive := first (keys (groupByLabel .Containers "com.github.nginx-proxy.nginx-proxy.keepalive")) }}
    location {{ .Path }} {
        {{- if eq .NetworkTag "internal" }}
        # Only allow traffic from internal clients
        include /etc/nginx/network_internal.conf;
        {{- end }}

        {{- if eq .Proto "uwsgi" }}
        include uwsgi_params;
        uwsgi_pass {{ trim .Proto }}://{{ trim .Upstream }};
        {{- else if eq .Proto "fastcgi" }}
        root {{ trim .VhostRoot }};
        include fastcgi_params;
        fastcgi_pass {{ trim .Upstream }};
            {{- if $keepalive }}
        fastcgi_keep_conn on;
            {{- end }}
        {{- else if eq .Proto "grpc" }}
        grpc_pass {{ trim .Proto }}://{{ trim .Upstream }};
        {{- else }}
        proxy_pass {{ trim .Proto }}://{{ trim .Upstream }}{{ trim .Dest }};
        set $upstream_keepalive {{ if $keepalive }}true{{ else }}false{{ end }};
        {{- end }}

        {{- if .Http3 }}
        add_header Alt-Svc 'h3=":443"; ma=86400';
        {{- end }}

        {{- if (exists (printf "/etc/nginx/htpasswd/%s" .Host)) }}
        auth_basic "Restricted {{ .Host }}";
        auth_basic_user_file {{ (printf "/etc/nginx/htpasswd/%s" .Host) }};
        {{- end }}

        {{- if (exists (printf "/etc/nginx/vhost.d/%s_%s_location" .Host (sha1 .Path) )) }}
        include {{ printf "/etc/nginx/vhost.d/%s_%s_location" .Host (sha1 .Path) }};
        {{- else if (exists (printf "/etc/nginx/vhost.d/%s_location" .Host)) }}
        include {{ printf "/etc/nginx/vhost.d/%s_location" .Host}};
        {{- else if (exists "/etc/nginx/vhost.d/default_location") }}
        include /etc/nginx/vhost.d/default_location;
        {{- end }}
    }
    {{- end }}
{{- end }}

{{- define "upstream" }}
upstream {{ .Upstream }} {
    {{- $server_found := false }}
    {{- $loadbalance := first (keys (groupByLabel .Containers "com.github.nginx-proxy.nginx-proxy.loadbalance")) }}
    {{- if $loadbalance }}
    # From the container's loadbalance label:
    {{ $loadbalance }}
    {{- end }}
    {{- range $container := .Containers }}
    # Container: {{ $container.Name }}
        {{- $args := dict "globals" $.globals "container" $container }}
        {{- template "container_ip" $args }}
        {{- $ip := $args.ip }}
        {{- $args := dict "container" $container }}
        {{- template "container_port" $args }}
        {{- $port := $args.port }}
        {{- if $ip }}
            {{- $server_found = true }}
    server {{ $ip }}:{{ $port }};
        {{- end }}
    {{- end }}
    {{- /* nginx-proxy/nginx-proxy#1105 */}}
    {{- if not $server_found }}
    # Fallback entry
    server 127.0.0.1 down;
    {{- end }}
    {{- $keepalive := first (keys (groupByLabel .Containers "com.github.nginx-proxy.nginx-proxy.keepalive")) }}
    {{- if $keepalive }}
    keepalive {{ $keepalive }};
    {{- end }}
}
{{- end }}

# If we receive X-Forwarded-Proto, pass it through; otherwise, pass along the
# scheme used to connect to this server
map $http_x_forwarded_proto $proxy_x_forwarded_proto {
    default {{ if $globals.trust_downstream_proxy }}$http_x_forwarded_proto{{ else }}$scheme{{ end }};
    '' $scheme;
}

map $http_x_forwarded_host $proxy_x_forwarded_host {
    default {{ if $globals.trust_downstream_proxy }}$http_x_forwarded_host{{ else }}$http_host{{ end }};
    '' $http_host;
}

# If we receive X-Forwarded-Port, pass it through; otherwise, pass along the
# server port the client connected to
map $http_x_forwarded_port $proxy_x_forwarded_port {
    default {{ if $globals.trust_downstream_proxy }}$http_x_forwarded_port{{ else }}$server_port{{ end }};
    '' $server_port;
}

# If the request from the downstream client has an "Upgrade:" header (set to any
# non-empty value), pass "Connection: upgrade" to the upstream (backend) server.
# Otherwise, the value for the "Connection" header depends on whether the user
# has enabled keepalive to the upstream server.
map $http_upgrade $proxy_connection {
    default upgrade;
    '' $proxy_connection_noupgrade;
}
map $upstream_keepalive $proxy_connection_noupgrade {
    # Preserve nginx's default behavior (send "Connection: close").
    default close;
    # Use an empty string to cancel nginx's default behavior.
    true '';
}
# Abuse the map directive (see <https://stackoverflow.com/q/14433309>) to ensure
# that $upstream_keepalive is always defined.  This is necessary because:
#   - The $proxy_connection variable is indirectly derived from
#     $upstream_keepalive, so $upstream_keepalive must be defined whenever
#     $proxy_connection is resolved.
#   - The $proxy_connection variable is used in a proxy_set_header directive in
#     the http block, so it is always fully resolved for every request -- even
#     those where proxy_pass is not used (e.g., unknown virtual host).
map "" $upstream_keepalive {
    # The value here should not matter because it should always be overridden in
    # a location block (see the "location" template) for all requests where the
    # value actually matters.
    default false;
}

# Apply fix for very long server names
server_names_hash_bucket_size 128;

# Default dhparam
{{- if (exists "/etc/nginx/dhparam/dhparam.pem") }}
ssl_dhparam /etc/nginx/dhparam/dhparam.pem;
{{- end }}

# Set appropriate X-Forwarded-Ssl header based on $proxy_x_forwarded_proto
map $proxy_x_forwarded_proto $proxy_x_forwarded_ssl {
    default off;
    https on;
}

gzip_types text/plain text/css application/javascript application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;

log_format vhost '{{ or $globals.Env.LOG_FORMAT "$host $remote_addr - $remote_user [$time_local] \"$request\" $status $body_bytes_sent \"$http_referer\" \"$http_user_agent\" \"$upstream_addr\"" }}';

access_log off;

{{- template "ssl_policy" (dict "ssl_policy" $globals.ssl_policy) }}
error_log /dev/stderr;

{{- if $globals.Env.RESOLVERS }}
resolver {{ $globals.Env.RESOLVERS }};
{{- end }}

{{- if (exists "/etc/nginx/proxy.conf") }}
include /etc/nginx/proxy.conf;
{{- else }}
# HTTP 1.1 support
proxy_http_version 1.1;
proxy_buffering off;
proxy_set_header Host $http_host;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $proxy_connection;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $proxy_x_forwarded_host;
proxy_set_header X-Forwarded-Proto $proxy_x_forwarded_proto;
proxy_set_header X-Forwarded-Ssl $proxy_x_forwarded_ssl;
proxy_set_header X-Forwarded-Port $proxy_x_forwarded_port;
proxy_set_header X-Original-URI $request_uri;

# Mitigate httpoxy attack (see README for details)
proxy_set_header Proxy "";
{{- end }}

{{- /*
     * Precompute some information about each vhost.  This is done early because
     * the creation of fallback servers depends on DEFAULT_HOST, HTTPS_METHOD,
     * and whether there are any missing certs.
     */}}
{{- range $vhost, $containers := groupByMulti $globals.containers "Env.VIRTUAL_HOST" "," }}
    {{- $vhost := trim $vhost }}
    {{- if not $vhost }}
        {{- /* Ignore containers with VIRTUAL_HOST set to the empty string. */}}
        {{- continue }}
    {{- end }}
    {{- $certName := first (groupByKeys $containers "Env.CERT_NAME") }}
    {{- $vhostCert := closest (dir "/etc/nginx/certs") (printf "%s.crt" $vhost) }}
    {{- $vhostCert = trimSuffix ".crt" $vhostCert }}
    {{- $vhostCert = trimSuffix ".key" $vhostCert }}
    {{- $cert := or $certName $vhostCert }}
    {{- $cert_ok := and (ne $cert "") (exists (printf "/etc/nginx/certs/%s.crt" $cert)) (exists (printf "/etc/nginx/certs/%s.key" $cert)) }}
    {{- $default := eq $globals.Env.DEFAULT_HOST $vhost }}
    {{- $https_method := or (first (groupByKeys $containers "Env.HTTPS_METHOD")) $globals.Env.HTTPS_METHOD "redirect" }}
    {{- $_ := set $globals.vhosts $vhost (dict "cert" $cert "cert_ok" $cert_ok "containers" $containers "default" $default "https_method" $https_method) }}
{{- end }}

{{- /*
     * If needed, create a catch-all fallback server to send an error code to
     * clients that request something from an unknown vhost.
     *
     * This server must appear first in the generated config because nginx uses
     * the first `server` directive to handle requests that don't match any of
     * the other `server` directives.  An alternative approach would be to add
     * the `default_server` option to the `listen` directives inside this
     * `server`, but some users inject a custom `server` directive that uses
     * `default_server`.  Using `default_server` here would cause nginx to fail
     * to start for those users.  See
     * <https://github.com/nginx-proxy/nginx-proxy/issues/2212>.
     */}}
{{- block "fallback_server" $globals }}
    {{- $globals := . }}
    {{- $http_exists := false }}
    {{- $https_exists := false }}
    {{- $default_http_exists := false }}
    {{- $default_https_exists := false }}
    {{- range $vhost := $globals.vhosts }}
        {{- $http := or (ne $vhost.https_method "nohttp") (not $vhost.cert_ok) }}
        {{- $https := ne $vhost.https_method "nohttps" }}
        {{- $http_exists = or $http_exists $http }}
        {{- $https_exists = or $https_exists $https }}
        {{- $default_http_exists = or $default_http_exists (and $http $vhost.default) }}
        {{- $default_https_exists = or $default_https_exists (and $https $vhost.default) }}
    {{- end }}
    {{- $fallback_http := and $http_exists (not $default_http_exists) }}
    {{- $fallback_https := and $https_exists (not $default_https_exists) }}
    {{- /*
         * If there are no vhosts at all, create fallbacks for both plain http
         * and https so that clients get something more useful than a connection
         * refused error.
         */}}
    {{- if and (not $http_exists) (not $https_exists) }}
        {{- $fallback_http = true }}
        {{- $fallback_https = true }}
    {{- end }}
    {{- if or $fallback_http $fallback_https }}
server {
    server_name _; # This is just an invalid value which will never trigger on a real hostname.
    server_tokens off;
    http2 on;
    {{- if $globals.enable_http3 }}
    http3 on;
    {{- end }}
        {{- if $fallback_http }}
    listen {{ $globals.external_http_port }}; {{- /* Do not add `default_server` (see comment above). */}}
                {{- if $globals.enable_ipv6 }}
    listen [::]:{{ $globals.external_http_port }}; {{- /* Do not add `default_server` (see comment above). */}}
                {{- end }}
        {{- end }}
        {{- if $fallback_https }}
    listen {{ $globals.external_https_port }} ssl; {{- /* Do not add `default_server` (see comment above). */}}
            {{- if $globals.enable_http3 }}
    listen {{ $globals.external_https_port }} quic reuseport;
            {{- end }}
            {{- if $globals.enable_ipv6 }}
    listen [::]:{{ $globals.external_https_port }} ssl; {{- /* Do not add `default_server` (see comment above). */}}
            {{- end }}
            {{- if and $globals.enable_ipv6 $globals.enable_http3 }}
    listen [::]:{{ $globals.external_https_port }} quic reuseport; {{- /* Do not add `default_server` (see comment above). */}}
            {{- end }}
    ssl_session_cache shared:SSL:50m;
    ssl_session_tickets off;
        {{- end }}
    {{ $globals.access_log }}
        {{- if $globals.default_cert_ok }}
    ssl_certificate /etc/nginx/certs/default.crt;
    ssl_certificate_key /etc/nginx/certs/default.key;
        {{- else }}
    # No default.crt certificate found for this vhost, so force nginx to emit a
    # TLS error if the client connects via https.
    {{- /* See the comment in the main `server` directive for rationale. */}}
    ssl_ciphers aNULL;
    set $empty "";
    ssl_certificate data:$empty;
    ssl_certificate_key data:$empty;
    if ($https) {
        return 444;
    }
        {{- end }}
    return 503;
}
    {{- end }}
{{- end }}

{{- range $host, $vhost := $globals.vhosts }}
    {{- $cert := $vhost.cert }}
    {{- $cert_ok := $vhost.cert_ok }}
    {{- $containers := $vhost.containers }}
    {{- $default_server := when $vhost.default "default_server" "" }}
    {{- $https_method := $vhost.https_method }}

    {{- $is_regexp := hasPrefix "~" $host }}
    {{- $upstream_name := when (or $is_regexp $globals.sha1_upstream_name) (sha1 $host) $host }}

    {{- $paths := groupBy $containers "Env.VIRTUAL_PATH" }}
    {{- $nPaths := len $paths }}
    {{- if eq $nPaths 0 }}
        {{- $paths = dict "/" $containers }}
    {{- end }}

    {{- range $path, $containers := $paths }}
        {{- $upstream := $upstream_name }}
        {{- if gt $nPaths 0 }}
            {{- $sum := sha1 $path }}
            {{- $upstream = printf "%s-%s" $upstream $sum }}
        {{- end }}
# {{ $host }}{{ $path }}
{{ template "upstream" (dict "globals" $globals "Upstream" $upstream "Containers" $containers) }}
    {{- end }}

    {{- /*
         * Get the SERVER_TOKENS defined by containers w/ the same vhost,
         * falling back to "".
         */}}
    {{- $server_tokens := trim (or (first (groupByKeys $containers "Env.SERVER_TOKENS")) "") }}

    {{- /*
         * Get the SSL_POLICY defined by containers w/ the same vhost, falling
         * back to empty string (use default).
         */}}
    {{- $ssl_policy := or (first (groupByKeys $containers "Env.SSL_POLICY")) "" }}

    {{- /*
         * Get the HSTS defined by containers w/ the same vhost, falling back to
         * "max-age=31536000".
         */}}
    {{- $hsts := or (first (groupByKeys $containers "Env.HSTS")) (or $globals.Env.HSTS "max-age=31536000") }}

    {{- /* Get the VIRTUAL_ROOT By containers w/ use fastcgi root */}}
    {{- $vhost_root := or (first (groupByKeys $containers "Env.VIRTUAL_ROOT")) "/var/www/public" }}

    {{- if and $cert_ok (eq $https_method "redirect") }}
server {
    server_name {{ $host }};
        {{- if $server_tokens }}
    server_tokens {{ $server_tokens }};
        {{- end }}
    listen {{ $globals.external_http_port }} {{ $default_server }};
        {{- if $globals.enable_ipv6 }}
    listen [::]:{{ $globals.external_http_port }} {{ $default_server }};
        {{- end }}
    {{ $globals.access_log }}

    # Do not HTTPS redirect Let's Encrypt ACME challenge
    location ^~ /.well-known/acme-challenge/ {
        auth_basic off;
        auth_request off;
        allow all;
        root /usr/share/nginx/html;
        try_files $uri =404;
        break;
    }

    location / {
        {{- if eq $globals.external_https_port "443" }}
        return 301 https://$host$request_uri;
        {{- else }}
        return 301 https://$host:{{ $globals.external_https_port }}$request_uri;
        {{- end }}
    }
}
    {{- end }}

server {
    server_name {{ $host }};
    {{- if $server_tokens }}
    server_tokens {{ $server_tokens }};
    {{- end }}
    http2 on;
    {{- if $globals.enable_http3 }}
    http3 on;
    {{- end }}
    {{ $globals.access_log }}
    {{- if or (eq $https_method "nohttps") (not $cert_ok) (eq $https_method "noredirect") }}
    listen {{ $globals.external_http_port }} {{ $default_server }};
        {{- if $globals.enable_ipv6 }}
    listen [::]:{{ $globals.external_http_port }} {{ $default_server }};
        {{- end }}
    {{- end }}
    {{- if ne $https_method "nohttps" }}
    listen {{ $globals.external_https_port }} ssl {{ $default_server }};
        {{- if $globals.enable_http3 }}
    listen {{ $globals.external_https_port }} quic {{ $default_server }};
        {{- end }}
        {{- if $globals.enable_ipv6 }}
    listen [::]:{{ $globals.external_https_port }} ssl {{ $default_server }};
        {{- end }}
        {{- if and $globals.enable_ipv6 $globals.enable_http3 }}
    listen [::]:{{ $globals.external_https_port }} quic {{ $default_server }};
        {{- end }}

        {{- if $cert_ok }}
            {{- template "ssl_policy" (dict "ssl_policy" $ssl_policy) }}

    ssl_session_timeout 5m;
    ssl_session_cache shared:SSL:50m;
    ssl_session_tickets off;

    ssl_certificate /etc/nginx/certs/{{ (printf "%s.crt" $cert) }};
    ssl_certificate_key /etc/nginx/certs/{{ (printf "%s.key" $cert) }};

            {{- if (exists (printf "/etc/nginx/certs/%s.dhparam.pem" $cert)) }}
    ssl_dhparam {{ printf "/etc/nginx/certs/%s.dhparam.pem" $cert }};
            {{- end }}

            {{- if (exists (printf "/etc/nginx/certs/%s.chain.pem" $cert)) }}
    ssl_stapling on;
    ssl_stapling_verify on;
    ssl_trusted_certificate {{ printf "/etc/nginx/certs/%s.chain.pem" $cert }};
            {{- end }}

            {{- if (not (or (eq $https_method "noredirect") (eq $hsts "off"))) }}
    set $sts_header "";
    if ($https) {
        set $sts_header "{{ trim $hsts }}";
    }
    add_header Strict-Transport-Security $sts_header always;
            {{- end }}
        {{- else if $globals.default_cert_ok }}
    # No certificate found for this vhost, so use the default certificate and
    # return an error code if the user connects via https.
    ssl_certificate /etc/nginx/certs/default.crt;
    ssl_certificate_key /etc/nginx/certs/default.key;
    if ($https) {
        return 500;
    }
        {{- else }}
    # No certificate found for this vhost, so force nginx to emit a TLS error if
    # the client connects via https.
            {{- /*
                 * The alternative is to not provide an https server for this
                 * vhost, which would either cause the user to see the wrong
                 * vhost (if there is another vhost with a certificate) or a
                 * connection refused error (if there is no other vhost with a
                 * certificate).  A TLS error is easier to troubleshoot, and is
                 * safer than serving the wrong vhost.  Also see
                 * <https://serverfault.com/a/1044022>.
                 */}}
    ssl_ciphers aNULL;
    set $empty "";
    ssl_certificate data:$empty;
    ssl_certificate_key data:$empty;
    if ($https) {
        return 444;
    }
        {{- end }}
    {{- end }}

    {{- if (exists (printf "/etc/nginx/vhost.d/%s" $host)) }}
    include {{ printf "/etc/nginx/vhost.d/%s" $host }};
    {{- else if (exists "/etc/nginx/vhost.d/default") }}
    include /etc/nginx/vhost.d/default;
    {{- end }}

    {{- range $path, $containers := $paths }}
        {{- /*
             * Get the VIRTUAL_PROTO defined by containers w/ the same
             * vhost-vpath, falling back to "http".
             */}}
        {{- $proto := trim (or (first (groupByKeys $containers "Env.VIRTUAL_PROTO")) "http") }}

        {{- /*
             * Get the NETWORK_ACCESS defined by containers w/ the same vhost,
             * falling back to "external".
             */}}
        {{- $network_tag := or (first (groupByKeys $containers "Env.NETWORK_ACCESS")) "external" }}
        {{- $upstream := $upstream_name }}
        {{- $dest := "" }}
        {{- if gt $nPaths 0 }}
            {{- $sum := sha1 $path }}
            {{- $upstream = printf "%s-%s" $upstream $sum }}
            {{- $dest = (or (first (groupByKeys $containers "Env.VIRTUAL_DEST")) "") }}
        {{- end }}
        {{- template "location" (dict "Path" $path "Proto" $proto "Upstream" $upstream "Host" $host "VhostRoot" $vhost_root "Dest" $dest "NetworkTag" $network_tag "Containers" $containers "Http3" $globals.enable_http3) }}
    {{- end }}
    {{- if and (not (contains $paths "/")) (ne $globals.default_root_response "none")}}
    location / {
        return {{ $globals.default_root_response }};
    }
    {{- end }}
}
{{- end }}

[SchoNie]

First: Good to mention for other testers: the quic port is UDP so has to be explicitly added in the docker compose file (443:443/udp) as it defaults to TCP.

I am having the same result as @buchdag error 400 but with a little more context: 400 Bad Request: missing required Host header* Connection #0 to host foo.bar.com left intact
Adding the host header in curl makes no difference.
I see in access log the Host value is "-" don't know why...
http2 is working OK.

curl.exe -v -k --http3 https://foo.bar.com/
*   Trying 172.24.77.62:443...
* Connect socket 360 over QUIC to 172.24.77.62:443
* Skipped certificate verification
* Connected to test.nh-at5.nl (172.24.77.62) port 443 (#0)
* h2h3 [:method: GET]
* h2h3 [:path: /]
* h2h3 [:scheme: https]
* h2h3 [:authority: foo.bar.com]
* h2h3 [user-agent: curl/7.86.0]
* h2h3 [accept: */*]
* Using HTTP/3 Stream ID: 0 (easy handle 0x1ae4e9874a0)
> GET / HTTP/3
> Host: foo.bar.com
> user-agent: curl/7.86.0
> accept: */*
>
< HTTP/3 400
< server: nginx/1.25.1
< date: Fri, 14 Jul 2023 09:19:09 GMT
< content-type: text/plain; charset=utf-8
<
400 Bad Request: missing required Host header* Connection #0 to host foo.bar.com left intact

[SchoNie]

I tried some more, as the error pointed me to the Host header. I commented #proxy_set_header Host $http_host;
And now it's working. But still no clear explanation for me.

[buchdag]

Thanks for the investigation.

So its seems we can't have both HTTP/3 and HTTP/2 HTTP/1.1 correctly reverse proxying a given host at the same time ?

[SchoNie]

Yes, all HTTP/3 and HTTP/2 HTTP/1.1 are OK. I can confirm the alt-svc header makes Chrome automatically use quic.

I found a ticket: The value of variable $http_host will not fallback to the value of :authority pseudo-header when the client not providing the request header Host which seems to be the reason.

Avoid using $http_host, it returns the value of the Host header, and not the :authority pseudo-header (or the authority component from the request line in HTTP/1.x). Use $host instead, it represents the host value nginx is actually using from the request.

So I changed proxy_set_header Host $http_host; to proxy_set_header Host $host; and this was succesfull.
Is there any reason $http_host is used? Looking back in commit history I see this is here for a long time. Can we propose to change this?
whats-the-difference-of-host-and-http-host-in-nginx

Do you want me to open a PR to add http3 (maybe hide this feature behind a docker label boolean)?

[buchdag]

Is there any reason $http_host is used ?

Not that I'm aware of 😅

Do you want me to open a PR to add HTTP/3 (maybe hide this feature behind a docker label boolean)?

I had this planned and almost ready on another machine, including optional disabling of HTTP/2, let me just fetch the diff so we can have a starting point.

[Knapoc]

that makes sense why it was working for me - I have changed the header in proxy.conf some time ago.

Do you guys get the Strict-Transport-Security-header applied when enabling HTTP/3?

Edit: Ah never mind... The nginx documentation states this

There could be several add_header directives. These directives are inherited from the previous configuration level if and only if there are no add_header directives defined on the current level.

[buchdag]

Draft PR: #2278

[QuAzI]

Nginx 1.25.2 released with some fixes for HTTP/3

[buchdag]

Feature has been merged and is available.

[EarthCompass]

Hello. After upgrading nginx-proxy from 1.3.1 to 1.4.0, my forwarding to the code-server stopped working due to inconsistency between Origin and Host header (in HTTP1.1).

v1.3.1, works fine

                                                                                                  
 client      ------------>          nginx-proxy          -------->           code-server          
            Host: coder:8080                         Host: coder:8080                             
            Origin: http://coder:8080                Origin: http://coder:8080                    
                                                     X-Forwarded-Host: coder:8080                 
                                                     X-Forwarded-Port: 8080                       
                                                                                                  
                                                                                                  
                                                                                                  

v1.4.0, error with 403 Forbidden due to Origin check
https://github.com/coder/code-server/blob/fdb46d3fe5808f205488abcc5b8104a5ecf664ae/src/node/http.ts#L362-L395

                                                                                                  
                                                                                                  
 client      ------------>          nginx-proxy          -------->           code-server          
            Host: coder:8080                         Host: coder                                  
            Origin: http://coder:8080                Origin: http://coder:8080                    
                                                     X-Forwarded-Host: coder                      
                                                     X-Forwarded-Port: 8080                       
                                                                                                  

I'm not sure which version has the right behavior, or maybe the coder-server should take X-Forwarded-Port header into consideration?

[SchoNie]

1.3.1 was using $http_host and this was changed in 1.4.0 to $host
$http_host can have the port number added in some cases I read. So that is probably why the check fails.

Are you setting the "Origin" header yourself? Try to change that also to $host
Or if I read the code-server manual correctly that Origin header is unsafe and now you can configure by CLI or Environment which proxies are trusted (so no need to set the Origin header)

[buchdag]

On the Host header, the port is optional, so both Host: coder:8080 and Host: coder are valid.

The X-Forwarded-Host header should only contain the host, so X-Forwarded-Host: coder:8080 was incorrect.

maybe the coder-server should take X-Forwarded-Port header into consideration ?

Honestly I don't know which is the best practice here, but adding back the port to the Host header of the proxy shouldn't be too difficult. I'll look into it.

edit : ninjaed by @SchoNie

[EarthCompass]

Thanks for your replies.

Are you setting the "Origin" header yourself?

The "Origin" header is set by the frontend js (or the browser).

Honestly I don't know which is the best practice here, but adding back the port to the Host header of the proxy shouldn't be too difficult. I'll look into it.

These non-standard headers are truly confusing 😢 .

I'll use v1.3.1 to avoid this issue for now.