Netbird Proxy TCP forwarding broke with 0.72.3

[Kariton]

Before posting

• I searched existing discussions and issues, including closed ones, and checked the relevant docs.
• I believe this is a product bug rather than a configuration or setup question.
• I can reproduce this issue, or for intermittent issues I've included trigger, frequency, and timing details below.
• I removed or anonymized sensitive data from logs, screenshots, and configuration.

Affected area

Reverse Proxy

Deployment type

Self-hosted - advanced/custom deployment

Operating system or environment

Linux, Docker

NetBird version and upgrade status

Updated Netbird Proxy

• from: ghcr.io/netbirdio/reverse-proxy:0.72.2@sha256:4921852487fb3384ee3287dafbfb04bf0f979cc2c1bf6c30dfa238171003a99c
• to: ghcr.io/netbirdio/reverse-proxy:0.72.3@sha256:58c3b6b4bfa109e610c4e4f83b14eb2d71194b720efcde20f90c75e727191a89

Did this work before?

Yes, this worked before

Regression details

Netbird Proxy does no longer open a listener for configured TCP port after updating to 0.72.3
Reverting back to 0.72.2 works.

Summary

Create Reverse Proxy Service:

• somehost.example.com
• Service Type: TCP
• select applicable peer
• Listen Port: 2222
• (Target) Port: 2222

Netbird Proxy does no longer start the TCP listener at :2222 - which used to work prior to the update, reverting back to 0.72.2 works again.

Current behavior

v0.72.3:

# ss -tulpn  | grep netbird-proxy                                                                                                                                                                                                     
udp   UNCONN 0      0            0.0.0.0:59467      0.0.0.0:*    users:(("netbird-proxy",pid=323890,fd=20))                                                                                                                                                           
udp   UNCONN 0      0               [::]:59467         [::]:*    users:(("netbird-proxy",pid=323890,fd=21))                                                                                                                                                           
tcp   LISTEN 0      4096      172.18.0.1:8443       0.0.0.0:*    users:(("netbird-proxy",pid=323890,fd=11))                                                                                                                                                           
tcp   LISTEN 0      4096       127.0.0.1:8080       0.0.0.0:*    users:(("netbird-proxy",pid=323890,fd=10))                                                                                                                                                           
tcp   LISTEN 0      4096       127.0.0.1:6060       0.0.0.0:*    users:(("netbird-proxy",pid=323890,fd=7))

Expected behavior

v0.72.2:

 ss -tulpn  | grep netbird-proxy
udp   UNCONN 0      0            0.0.0.0:44036      0.0.0.0:*    users:(("netbird-proxy",pid=332142,fd=20))
udp   UNCONN 0      0               [::]:44036         [::]:*    users:(("netbird-proxy",pid=332142,fd=21))
tcp   LISTEN 0      4096      172.18.0.1:8443       0.0.0.0:*    users:(("netbird-proxy",pid=332142,fd=11))
tcp   LISTEN 0      4096       127.0.0.1:8080       0.0.0.0:*    users:(("netbird-proxy",pid=332142,fd=10))
tcp   LISTEN 0      4096       127.0.0.1:6060       0.0.0.0:*    users:(("netbird-proxy",pid=332142,fd=7)) 
tcp   LISTEN 0      4096               *:2222             *:*    users:(("netbird-proxy",pid=332142,fd=12))

Steps to reproduce

Expose TCP service via Netbird Proxy.

Environment and topology

compose snipped:

  netbird-proxy:
    #
    # The Proxy service is expected to request LE certificates for all kinds of custom domains you may add in NetBird.
    # With that in mind it is also recommended to use TCP routes instead of HTTP.
    # For various reasons thats not great:
    #   - LE certificate limit of 50/domain/7days
    #   - LE request limit of 5/identifiers/7days
    #   - Must use valid domain; HTTP or TLS-ALPN-01 challange required
    #   -> https://letsencrypt.org/docs/rate-limits/
    #
    # I dont like that so ill give him a static self-signed one; a certificate is required.
    # Then ill use a HTTP route, which validates the custom backend certificate, with publicdomain matching wildcard certificate in Traefik.
    # Another benifit is that NetBird Proxy access is now logged in Traefik and can utilise CrowdSec and Anubis.
    #
    #   openssl req -x509 -newkey ec -pkeyopt ec_paramgen_curve:P-256 -sha256 -days 36500 -nodes \
    #     -keyout ./data/netbird-proxy/tls.key \
    #     -out ./data/netbird-proxy/tls.crt \
    #       -subj "/CN=netbird-proxy" \
    #     -addext "subjectAltName=DNS:netbird-proxy,IP:172.18.0.1" \
    #     -addext "basicConstraints=critical,CA:FALSE" \
    #     -addext "keyUsage=critical,digitalSignature" \
    #     -addext "extendedKeyUsage=serverAuth"
    # 
    #   chown 1000:1000 data/netbird-proxy/tls.*
    #   chmod 0400 data/netbird-proxy/tls.key
    #   chmod 0444 data/netbird-proxy/tls.crt
    #
    container_name: netbird-proxy
    image: ghcr.io/netbirdio/reverse-proxy:0.72.2@sha256:4921852487fb3384ee3287dafbfb04bf0f979cc2c1bf6c30dfa238171003a99c
    restart: unless-stopped
    depends_on:
      netbird-server:
        condition: service_started
    environment:
      NB_PROXY_MANAGEMENT_ADDRESS: https://vpn.example.com
      NB_PROXY_ADDRESS: 172.18.0.1:8443  # host loopback/pod net; traefik terminates TLS and  has app-sec and anubis in front of netbird
      NB_PROXY_DOMAIN: example.com
      NB_PROXY_TOKEN: REDACTED
      NB_PROXY_TRUSTED_PROXIES: 172.18.0.10/32  # traefik
      NB_PROXY_CROWDSEC_API_URL: http://127.0.0.1:8081
      NB_PROXY_CROWDSEC_API_KEY: REDACTED
    volumes:
      - ./data/netbird-proxy:/certs  # dummy certificate between traefik and netbird
    network_mode: host

Self-hosted details, if available

No response

Logs, status output, or debug evidence

v0.72.2:

[...]
DEBG [type: UPDATE_TYPE_CREATED, domain: somehost.example.com, id: d8ku5ar5h8ic73ep8mq0, mapping: {"type":"UPDATE_TYPE_CREATED", "id":"d8ku5ar5h8ic73ep8mq0", "account_id":"d7d4vub5h8ic73db2d50", "domain":"somehost.example.com", "path":[{"path":"", "target":"10.97.38.94:2222", "options":null}], "auth_token":"[REDACTED]", "auth":{"session_key":"k8iapA4Z5kfq9G420FuZQqZy3zf3DOJEdnBFcLEH0Ck=", "max_session_age_seconds":"86400", "password":false, "pin":false, "oidc":false, "header_auths":[]}, "pass_host_header":false, "rewrite_redirects":false, "mode":"tcp", "listen_port":2222, "access_restrictions":{"allowed_cidrs":[], "blocked_cidrs":[], "allowed_countries":["DE"], "blocked_countries":[], "crowdsec_mode":"enforce"}, "private":false}] proxy/server.go:1472: Processing mapping update
DEBG [service_key: l4:d8ku5ar5h8ic73ep8mq0, account_id: d7d4vub5h8ic73db2d50] proxy/internal/roundtrip/netbird.go:214: registered service with existing client
DEBG proxy/server.go:1114: started per-port router on :2222
DEBG proxy/server.go:1311: Processing mapping update completed
INFO proxy/server.go:1367: Initial mapping sync complete

v0.72.3:

DEBG [type: UPDATE_TYPE_CREATED, domain: somehost.example.com, id: d8ku5ar5h8ic73ep8mq0, mapping: {"type":"UPDATE_TYPE_CREATED","id":"d8ku5ar5h8ic73ep8mq0","account_id":"d7d4vub5h8ic73db2d50","domain":"somehost.example.com","path":[
{"path":"","target":"10.97.38.94:2222","options":null}],"auth_token":"[REDACTED]","auth":{"session_key":"k8iapA4Z5kfq9G420FuZQqZy3zf3DOJEdnBFcLEH0Ck=","max_session_age_seconds":"86400","password":false,"pin":false,"oidc":false,"header_auths":[]},"pass_host_heade
r":false,"rewrite_redirects":false,"mode":"tcp","listen_port":2222,"access_restrictions":{"allowed_cidrs":[],"blocked_cidrs":[],"allowed_countries":["DE"],"blocked_countries":[],"crowdsec_mode":"enforce"},"private":false}] proxy/server.go:1532: Processing mappin
g update
DEBG [account_id: d7d4vub5h8ic73db2d50, service_key: l4:d8ku5ar5h8ic73ep8mq0] proxy/internal/roundtrip/netbird.go:281: registered service with existing client
DEBG proxy/server.go:1124: started per-port router on :2222
DEBG proxy/server.go:1331: Processing mapping update completed
INFO proxy/server.go:1387: Initial mapping sync complete

### Related issues or discussions

_No response_

### Impact

_No response_

### Additional context

_No response_

[Kariton]

with v0.72.4:

# ss -tulpn | grep netbird-proxy                                                                                                                                         
udp   UNCONN 0      0            0.0.0.0:55357      0.0.0.0:*    users:(("netbird-proxy",pid=929013,fd=19))                                                                                              
udp   UNCONN 0      0               [::]:55357         [::]:*    users:(("netbird-proxy",pid=929013,fd=20))                                                                                              
tcp   LISTEN 0      4096      172.18.0.1:8443       0.0.0.0:*    users:(("netbird-proxy",pid=929013,fd=10))                                                                                              
tcp   LISTEN 0      4096       127.0.0.1:6060       0.0.0.0:*    users:(("netbird-proxy",pid=929013,fd=4))                                                                                               
tcp   LISTEN 0      4096       127.0.0.1:8080       0.0.0.0:*    users:(("netbird-proxy",pid=929013,fd=7))

reverted back to v0.72.2.

[Kariton]

#6400