WebSocket upgrade headers stripped/mishandled by reverse proxy (v0.71.2) — silent failure, not the same as #5329/#5507

[sdejongh]

Before posting

• I searched existing discussions and issues, including closed ones, and checked the relevant docs.
• I believe this is a product bug rather than a configuration or setup question.
• I can reproduce this issue, or for intermittent issues I've included trigger, frequency, and timing details below.
• I removed or anonymized sensitive data from logs, screenshots, and configuration.

Affected area

Reverse Proxy

Deployment type

Self-hosted - quickstart script

Operating system or environment

Linux

NetBird version and upgrade status

• Management: v0.71.2 (netbird-porxy: v0.71.2, trefik 3.6.17)
• Dashboard: v2.38.0

Did this work before?

No, this never worked

Regression details

No response

Summary

The NetBird built-in reverse proxy (v0.71.0) silently drops WebSocket upgrade headers when forwarding HTTP/1.1 requests, and mishandles WebSocket-over-HTTP/2 requests (RFC 8441). The result is that no WebSocket-using application works behind the proxy: GitLab ActionCable, Immich, Home Assistant, Mattermost, Nextcloud Talk, and any other modern self-hosted web app.The failure is silent at the proxy level — no log entries, no warnings, no errors. Operators only see a 404 in the dashboard "Proxy Events" tab and reasonably assume their backend is misconfigured.

Current behavior

WebSocket handshake requests reaching netbird-proxy fail in two distinct ways depending on HTTP version:

HTTP/1.1: the Connection: Upgrade and Upgrade: websocket headers are silently stripped before forwarding to the backend. The upstream service receives a plain GET request on the WebSocket endpoint and returns whatever it normally returns for a non-WebSocket GET on that path (typically 404).

HTTP/2 (browser default via ALPN): the upstream nginx receives the request but rejects it with 400 websocket upgrade required, indicating that something in the HTTP/2 → HTTP/1.1 conversion (or RFC 8441 :protocol: websocket handling) is broken.

In both cases:

netbird-proxy container logs are silent — no warning, no error.
The dashboard "Proxy Events" tab shows the request as 404 Request failed with no further detail.
The retry loop from the client (typical for ActionCable, Socket.IO, etc.) produces a continuous stream of failed requests with no diagnostic information.

Expected behavior

netbird-proxy should detect WebSocket upgrade requests and proxy them correctly:

HTTP/1.1: preserve Connection: Upgrade and Upgrade: websocket when forwarding, and hijack the TCP connection to bridge frames between client and upstream.

HTTP/2: handle RFC 8441 CONNECT with :protocol: websocket, or downgrade to HTTP/1.1 toward the upstream while preserving the upgrade semantics.

The expected response to a valid WebSocket handshake is 101 Switching Protocols from the upstream, after which the proxy tunnels frames bidirectionally for the lifetime of the connection.

Steps to reproduce

1. Deploy self-hosted NetBird v0.71.2 via getting-started.sh (default stack: Traefik + netbird-proxy + netbird-server + dashboard + crowdsec).
2. Configure an HTTPS reverse proxy service in the dashboard pointing to any WebSocket-using backend. Tested case: self-hosted GitLab CE on http://:8929.
3. Open the application in a browser → DevTools → Network → WS tab. Observe repeated failed WebSocket handshakes on the application's WS endpoint (e.g. /-/cable for GitLab).
4. Reproduce with curl (see commands in the logs section).
5. Optionally bypass Traefik to confirm netbird-proxy is the cause — curl from inside the Docker network directly to netbird-proxy:8443 reproduces the bug identically.

Environment and topology

Browser (HTTP/2 via ALPN)
  → Internet
    → Traefik v3.6 on :443 (TCP/TLS passthrough by SNI)
      → netbird-proxy on :8443 (TLS termination, HTTP forwarding) ← bug here
        → GitLab nginx on http://<peer-ip>:8929 (via NetBird mesh)
          → GitLab Workhorse / Rails (ActionCable on /-/cable)

Self-hosted details, if available

• OS: Debian 12
• NetBird stackv0.71.2 (latest stable at time of report)
• Deployment: getting-started.sh

Logs, status output, or debug evidence

Logs:
(empty — no warning, no error, no entry related to the WebSocket path at default log level)

CURL TEST HTTP/1.1*

Command

curl -v -i --http1.1 \
  -H "Connection: Upgrade" -H "Upgrade: websocket" \
  -H "Sec-WebSocket-Version: 13" -H "Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==" \
  https://<service-fqdn>/-/cable

Response

> GET /-/cable HTTP/1.1
> Host: <service-fqdn>
> Connection: Upgrade
> Upgrade: websocket
> Sec-WebSocket-Version: 13
> Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==

< HTTP/1.1 404 Not Found
< Server: nginx
< X-Gitlab-Meta: {"correlation_id":"01KRX51R5DCBMVYEBDQSDWPJ9J","version":"1"}
< X-Request-Id: 01KRX51R5DCBMVYEBDQSDWPJ9J
< X-Runtime: 0.011231
< Content-Length: 14

Page not found

CURL TEST HTTP/2

Command

curl -v -i \
  -H "Connection: Upgrade" -H "Upgrade: websocket" \
  -H "Sec-WebSocket-Version: 13" -H "Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==" \
  https://<service-fqdn>/-/cable

Response

< HTTP/2 400 
< content-type: text/plain; charset=utf-8
< server: nginx
< content-length: 27

websocket upgrade required

By-PASSING Traefik

Command

docker run --rm --network netbird_netbird curlimages/curl:latest \
  curl -v -i -k --http1.1 \
  --resolve <service-fqdn>:8443:<netbird-proxy-ip> \
  -H "Connection: Upgrade" -H "Upgrade: websocket" \
  -H "Sec-WebSocket-Version: 13" -H "Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==" \
  https://<service-fqdn>:8443/-/cable

Response

< HTTP/1.1 404 Not Found
< Server: nginx
< X-Gitlab-Meta: {"correlation_id":"01KRX51R5DCBMVYEBDQSDWPJ9J","version":"1"}
< X-Runtime: 0.011231

Related issues or discussions

• Proxy does not support websocket #5329 : reported non-Hijacker ResponseWriter panics on v0.65.1 producing 502 errors with explicit log lines. Marked as fixed via [proxy] Support WebSocket #5312. The current bug is likely a regression or incomplete fix: the visible panic is gone, but the underlying WebSocket support was not actually implemented — the upgrade headers are now silently dropped instead of crashing the request.
• Failed to connect WebSocket when using reverse proxy #5507 : reports generic "WebSockets don't work" on v0.66.1 with OpenClaw and MaiBot WebUI. Same root cause area, different version, similar symptoms.

Impact

No response

Additional context

No response

[sdejongh]

Upgraded to latest 0.7.4, issue is still the same.

[lucasaffonso0]

Enabling this option will fix the problem:

[sdejongh]

This option is already enabled in all my tests.

[sdejongh]

I reinstalled Netbird from scratch (previous install was upgraded several times), it now seems to work.