STUNner v2.0: The Kubernetes real-time media gateway

We are proud to present STUNner v2.0.0, the next major release of the STUNner Kubernetes media gateway brought to you by l7mp.io. This release is the result of a year-long rewrite and stabilization work: a new dataplane, a new configuration API, a new release model, and beta support for SIP telephony, SRT live video and RTSP cameras besides WebRTC.

Major changes are the addition of a SIP gateway that brings VoIP telephony to Kubernetes; an SRT gateway to route live video streams to media servers, both in beta; Linux/eBPF kernel offload for every UDP protocol STUNner relays (L4, TURN, SRT, RTP); TURN/TCP support to allow TURN clients reach TCP backends that expose any TCP workload; the completely re-architected dataplane that lets us extend STUNner to other real-time communication protocols beyond TURN; plain L4 proxying over UDP, TCP, TLS and DTLS; a native TURN proxy that can route UDP/TCP clients to remote TURN server using the new TURNRoute resource; dual-stack IPv4/IPv6 relaying; post-quantum TLS cryptography support; a high-availability (HA) operator; and mediactl, a new tool to test media gateways end to end. Equally important, this release moves STUNner to an open-core model that lets us open up STUNner's well-established premium features, especially kernel offload, in the free tiers (limits apply).

The v2.0 release also brings breaking changes: please read the below before upgrading.

The open-core model

Up to v1.2, STUNner came in two flavors: the open-source release with its own installation mechanism, and the premium release with separate premium images and Helm chart. From v2.0 on there is only one STUNner.

• The source code of STUNner remains open-source software (OSS) distributed under the same permissive license as before: the dataplane (stunnerd), the gateway operator, the auth service, and the tooling (stunnerctl and mediactl). The dataplane is fully open source except the eBPF kernel offload engines (TC and XDP).
• The official container images (l7mp/stunnerd, l7mp/stunner-gateway-operator) are built by l7mp.io from the open-source code base plus the proprietary features, including the kernel offload engines and the Gateway API support for the SIP and SRT gateways. Without a license key the official images fall back to the OSS feature set, covering everything STUNner v1 did, plus kernel offload (limits apply).
• A customer key unlocks the premium features: user quotas, the DaemonSet dataplane, relay address discovery, dual-stack IPv4/IPv6 relaying, the HA operator and post-quantum TLS, plus higher limits for the beta SIP and SRT gateways and for the kernel offload. There are no limits on TURN at any tier. See l7mp.io/pricing for the limits of each tier and how to purchase a license, and the premium feature reference for the details.

Breaking changes

• The stunner-premium Helm chart and the l7mp/stunnerd-premium and l7mp/stunner-gateway-operator-premium images are discontinued. Install the usual stunner/stunner chart to get the official build: the default l7mp/stunnerd and l7mp/stunner-gateway-operator images carry the premium features, which can be unlocked by purchasing a customer key.
• A Gateway listener with a plain UDP, TCP, TLS or DTLS protocol is now an L4 proxy (see below). The plain protocol names are no longer aliases of the TURN protocols: use TURN-UDP, TURN-TCP, TURN-TLS or TURN-DTLS for TURN listeners.
• The legacy dataplane mode is gone for good: the --dataplane-mode operator flag, STUNNER_GATEWAY_OPERATOR_DATAPLANE_MODE environment variable, stunnerGatewayOperator.dataplane.mode chart value and the stunnerd-config ConfigMap are no longer supported. The operator always provisions a dataplane per Gateway and serves its config over the Config Discovery Service. The operator drops RBAC access to ConfigMaps (except for the leader election Role).
• The stunner.l7mp.io/v1alpha1 versions of GatewayConfig, Dataplane and StaticService are removed, only v1 is served. TURNRoute is served at stunner.l7mp.io/v1; only the experimental SIPTrunk is at v1alpha1. The Gateway API v1alpha2 UDPRoute and TCPRoute are no longer supported. Install the Gateway API CRDs shipped with the chart instead (v1.6).
• The deprecated auth type names plaintext and longterm, and the operator's timewindowed aliases, are gone: use static and ephemeral instead. GatewayConfig authType accepts static, ephemeral and none, the default is static.
• The operator flags --endpoint-discovery and --disable-endpointslice-controller are removed: backend endpoints always come from EndpointSlices, the Endpoints API fallback and the DNS fallback are gone.
• The stunnerd --force-ready-status flag is removed: a draining stunnerd always keeps passing the readiness probe, so kube-proxy keeps the UDP sessions of a terminating pod alive until they end. The terminating pod still gets no new connections. If a data plane rollout fails or hangs, the operator now sets the corresponding Gateway's Progammed status to false.
• The deprecated stunnerd --udp-thread-num (-u) flag is removed: UDP listeners are automatically scaled to GOMAXPROCS (the number of available CPU cores) and each new UDP connection gets an independent readloop. This ensures much better UDP scalability as before.
• The operator renders native v2 dataplane configs and the cluster names now carry the protocol of the route, like <namespace>/<route>/udp (/tcp for TCPRoutes, /turn for TURNRoutes). This changes the name label of the stunner_cluster_* Prometheus metrics: update your dashboards and alerts. Listener names (<namespace>/<gateway>/<listener>) are unchanged.
• The operator's leader election Lease is now named stunner-gateway-operator.l7mp.io. During a rolling update from an older operator the old and the new leader may overlap for a while: use the Recreate strategy for a clean handover.
• The venerable turncat tool is retired: its functionality is now absorbed by the tunnel mode of stunnerd.
• The Kubernetes addresses take the syntax k8s://<namespace>/<gateway>[/<listener>] for a Gateway listener (it used to be <gateway>:<listener>) and k8s://<namespace>/<service>:<port> for a peer, the same in stunnerd and mediactl.
• The TURNOffload license feature is retired: kernel offload is no longer a feature of the license but available on every tier, capped at the offload limit of the tier.
• The Go module path is now github.com/l7mp/stunner/v2.
• Gateway annotations are no longer copied to the stunnerd Deployment (Gateway labels still are): use the Dataplane annotations field to annotate the stunnerd pods.
• Removed: the post-install hook (cmd/post-install and its image), the static install manifests in deploy/manifests/static (install the CRDs from the chart instead), cmd/stunnerctl/stunnerctl.sh, and a couple of obsolete sample configs.

Deprecations

• The v1 stunnerd config API is deprecated. stunnerd, stunnerctl and the CDS client still load version: v1 configs and auto-upgrade to v2. The operator renders native v2 only. The v1 support will be removed in a later release.

A new dataplane

The stunnerd dataplane has gone through extensive rewrite and refactoring. The v2 data plane now consists of five kinds of objects: listeners accept client connections on UDP, TCP, TLS, DTLS and a couple of internal transports, servers implement the dataplane behavior (TURN, L4, SIP, RTP and SRT), clusters are the named sets of peer endpoints and transport, filters are ordered ALLOW/DENY rules on address blocks and ports, and auth policies hold the static or ephemeral credentials. Objects come and go independently and a missing object fails only the connections that need it, which makes reconfiguration fast and predictable. See the reference guides on the GatewayClass, Gateway and route resources have the details.

SIP and RTP (beta)

The new data plane architecture allowed us to extend STUNner beyond TURN to cover most of the protocols of the real-time communications landscape. The most important addition is support for VoIP telephony: STUNner's SIP gateways can now ingest SIP/RTP calls from carrier SIP trunks into your Kubernetes cluster. The stunnerd dataplane implements a transaction-stateful SIP proxy for inbound calls and an RTP media proxy, with built-in support for both popular trunk models: IP-authenticated trunks, where the carrier sends calls to the Gateway's public address, and registration-based trunks, where STUNner registers with the carrier's registrar and receives the calls over the registered connection. Digest authentication is available in all modes. You can turn a Gateway into a SIP gateway by naming a SIPTrunk resource in the Gateway's spec.infrastructure.parametersRef, and route the calls to your SIP servers and media servers with plain UDPRoutes. The SIP signaling and the media can go to the same server or to separate SIP and media server pools. A new SIP guide and a SIP echo tutorial have been added, and the Asterisk tutorial shows how to ingest SIP/RTP sesions for a real PBX.

The SIP gateway is in beta: the dataplane for SIP Gateways is restricted to a single pod, the API may change in any release, and there is no support at any tier. The number of SIP Gateways and parallel calls is limited per tier, see l7mp.io/pricing.

SRT (beta)

The new SRT gateway ingests SRT live video into Kubernetes: encoders, drones and cameras publish their streams to a single public UDP port, decoders and players read them from the same port, and STUNner routes every connection to the media server pod (e.g., MediaMTX) that holds its stream. STUNner reads the stream ID of the SRT handshake and sends every connection naming the same stream, publisher or reader, to the same pod, so the media servers can be scaled behind one address. Encryption stays end to end: STUNner never sees the passphrase. An SRT gateway is a Gateway with listeners of the new SRT protocol and UDPRoutes to the media servers. See the SRT guide, the SRT echo tutorial and the MediaMTX tutorial for how to configure STUNner over SRT.

Like the SIP gateway, the dataplane for SRT Gateways is restricted to a single pod and the SRT gateway is in beta with no support at any tier. The number of SRT Gateways and parallel streams is limited per tier; see l7mp.io/pricing for the details.

RTSP (beta)

STUNner can now front IP cameras: a media server deployed into a cluster (e.g., MediaMTX or go2rtc) pulls the camera streams, which can be watched over WebRTC through a TURN listener from browsers, or using native RTSP clients over a plain TCP listener. See the RTSP guide for the details.

RTSP is in beta too, and only RTSP interleaved over TCP is supported.

L4 proxying

Gateway listeners with a plain UDP, TCP, TLS or DTLS protocol now proxy each client connection into the cluster over the corresponding native transport (i.e., UDP, TCP, etc.), load-balanced in round robin over the endpoints of all TCP and UDP routes. This makes it possible to expose plain UDP or TCP services through the same Gateway as your TURN listeners. TLS and DTLS listeners terminate the encryption at STUNner. Note that plain listeners need a workload that authenticates its own clients (e.g., RTSP media server); see the security guide for the details.

TURN/TCP end-to-end

STUNner is no longer a UDP-only TURN gateway. TURN over TCP works end to end that lets TCP clients reach Kubernetes-bound TCP backends behind TURN. A TCPRoute attached to a TURN-TCP or TURN-TLS listener lets WebRTC clients reach TCP services in the cluster over RFC 6062 TCP allocations. TCPRoute support is now open source, and the official Gateway API TCPRoute works as well.

TURNRoute

The new TURNRoute resource routes the clients of a listener through an upstream TURN server: on a TURN listener every peer is reached through the upstream, on an L4 listener the route's backends are dialed through the TURN tunnel. This is the way to proxy connections to a remote TURN service, possibly in a different cluster. The CRD is optional: helm upgrade does not update CRDs, so on an upgraded installation apply the chart's CRDs by hand before using it.

Kernel offload for every protocol

The Linux/eBPF kernel offload, so far limited to TURN, has been extended to relay UDP flows for every protocol STUNner supports in the kernel: TURN/UDP channels, SRT streams, RTP/RTCP media, and plain UDP L4 flows. A more robust but somewhat slower TC/eBPF engine as well as an ultra-fast XDP engine are both available. Chiefly, kernel offload is now available at free tiers for everyone (limits apply). Use stunnerctl status to read the configured offload engine, the interfaces and the offloaded flows. See the kernel offload reference for more detail.

IPv6 and dual-stack (premium tiers)

STUNner now speaks IPv6 throughout the dataplane, the operator and the auth service. On dual-stack installations clients get a relay address of the address family they connected over, and the auth service hands out an ICE server for each public address of a Gateway.

Post-quantum TLS (premium tiers)

Post-quantum cryptography is now fully supported in STUNner: TURN-TLS listeners can now use a hybrid post-quantum key exchange. The stunner.l7mp.io/pqc-mode Gateway annotation selects the mode: enforced accepts only post-quantum key exchange, preferred prefers it but falls back to the classical key exchange protocols for legacy clients, and default leaves the defaults.

High-availability operator (premium tiers)

The gateway operator can now run in multiple replicas with leader election. A restart or a failover never pushes a partial config and the data plane works seamlessly across operator restarts and failovers.

Media traffic generators

The new mediactl tool sends packets and measures the returned flows over every protocol STUNner supports: SIP calls with RTP media, plain UDP and TCP, TURN relays (including RFC 6062 TCP relaying) and SRT. It measures packet loss, jitter, round trip time, and it can rate voice calls with the R-factor and the MOS of the ITU-T G.107 E-model. It simplifies Gateway and Service addressing with the usual k8s:// style URIs stunnerd has supported for long. See the mediactl guide for examples.

Miscellaneous improvements

As usual, this release comes with a number of fixes and improvements all around the place (see the above and the commit logs below). STUNner now supports Gateway addresses of type Hostname for TLS certificates issued to a DNS name, ships a hardened Config Discovery Service (CDS) client, supports a priorityClassName setting for the dataplane pods in the Dataplane resource as well as custom labels for the LoadBalancer Services in the GatewayConfig`, scales to very large clusters with hundreds of Gateways, reports the build version reported in the dataplane status and in the Prometheus metrics, and features a fully rewritten documentation with new user guides for WebRTC, SIP, SRT and RTSP and reworked tutorials. STUNner has moved to Go 1.27 and the latest Pion, Kubernetes and Gateway API dependencies.

We thank our users who contributed pull requests; STUNner would not be the same without your effort! Enjoy STUNner, join us at Discord, and don't forget to support us!

Commit logs

Dataplane

feat(api): Add dialing listeners, TLS blocks, SIP registrations
feat(api): Add plain-listener peer address, STDIN protocol, and TURN server SNI
feat(api): Add the SRT protocol
feat(api): Move the load-balancing policy to the L4 server
feat(cds): Let CDS discovery find the serving CDS server robustly
feat(cds): Let server Patch and de-duplicate per client
feat(config): Make auth a list of named policies
feat(config): Upper-case server types, reserve SIP and RTP
feat(dataplane): Rebuild on listeners, servers, clusters
feat(discovery): Export the auth service client
feat(filter): Add filters, make endpoints plain addresses
feat(flow): Offload a flow from the conntrack
feat(license): Add HAOperator and PQC to the catalog
feat(license): Add the SIP and SRT limits, print them in stunnerctl
feat(license): Drop TCPRoute and STUN from the catalog
feat(license): Publish the feature catalog
feat(license): Rename TURNOffload to KernelOffload
feat(license): Serve tier limits in the stunnerd status
feat(listener): Add listener labels to the conn tag
feat(listener): Add pqc_mode handling to the reconciliation logic
feat(mediactl): Add the mediactl library and command
feat(mediactl): Export the send workflow
feat(mediactl): Print packet and byte counts and rates with SI prefixes
feat(mediactl): Rate VoIP calls with the E-model
feat(offload): Admit the flows up to the offload limit
feat(offload): Report the running engine and its flows in the status
feat(quota): Move the user quota into open source
feat(rtp): Offload the media of the calls
feat(server): Deny unmatched permissions, balance L4 over clusters
feat(sip): Add the SIP and RTP servers
feat(sip): Advertise the listener's public address
feat(sip): Answer registrar digest challenges
feat(sip): Authenticate inbound requests by digest
feat(sip): Count calls per media proxy
feat(srt): Add the SRT server
feat(status): Report the total and the offloaded traffic
feat(stunnerd): Bring back the k8s:// URIs of the tunnel mode
feat(stunnerd): Take the offload interfaces on --offload
feat(tls): Move the PQC modes into open source
feat: Add optional public_addresses list to ListenerConfig
feat: Add stunnerd tunnel mode
feat: Add the L4 flow engine serving plain listeners
feat: Advertise a family-matched relay address (dual-stack)
feat: Dispatch plain listeners to the L4 flow engine
feat: Extend the v1.ClusterConfig with TURN-* relay protocols
feat: Implement the TURN relay cluster and refactor boilerplate
feat: Report offloaded traffic in listener/cluster telemetry
feat: Report the build in the status and Prom metrics
feat: Serve v2 configs in the CDS server
feat: Wire flow events into quota and offload accounting
fix(apis): Make DeepCopy exact
fix(auth): Give credentials without a user id the default user
fix(cds): Expand env vars on Load as Watch does
fix(ci): Build the icetester image on Go 1.27
fix(ci): Fetch tags so released artifacts carry a git version
fix(config): Keep watching a file after an invalid config
fix(config): Leave env-var substitution and validation to stunnerd
fix(discovery): Skip stunnerd pods that are not running
fix(license): Keep the limits off the network
fix(mediactl): Start the media of an answered call before the 2xx
fix(object): Report the session count and readiness in the status
fix(offload): Enforce the same offload matrix over all transports
fix(offload): Expose the offload engine's packet counter
fix(offload): Register only the flows the engines can accelerate
fix(siptx): Refuse requests without an RFC 3261 branch
fix(stunnerctl): Make sure icetest exits non-zero on failure
fix(stunnerd): Disable graceful shutdown in server and tunnel mode
fix(stunnerd): Relay to any peer in the standalone TURN server mode
fix(turn): Untrack client conns when they close
fix(whipconn): Close the conn of every WHIP request
fix: Add IPv6 support to admin object reconcilers
fix: Parse RFC 7065 TURN URIs and bracketed IPv6 hosts
fix: Refuse mixed transport relay configs
fix: Reject unbracketed IPv6 URIs deterministically
fix: use net.JoinHostPort to handle ipv6 addresses
perf(udp): Key the listener conns by netip.AddrPort
refactor(api): Remove the plaintext and longterm aliases
refactor(cli): Use the v2 protocols in the tools
refactor(flow): Move the conntrack to internal/flow
refactor(icetester): Move the ICE tester to pkg/icetester
refactor(k8s): One k8s:// address syntax for stunnerd and mediactl
refactor(object): Let Listener own its packet engine
refactor(relay): Let the upstream transports answer for themselves
refactor(stunnerd): Remove the udp-thread-num flag
refactor: Add boilerplate for SIP/RTP server to land
refactor: Collect the exported helpers under pkg/utils
refactor: Make bracketing IPv6 CDS server addrs mandatory
refactor: Rewrite the object system + implement TCP clusters, #232
refactor: Route turncat listener/peer parsing through ParseURI
refactor: Stack the relay transports as reusable connection wrappers
refactor: Unify listener and cluster protocols into v1.Protocol
chore(deps): Clear the golang.org/x security issues
chore(deps): Take the TURN channel accessors from upstream
chore(deps): Update dependencies and re-generate the openapi bindings
chore(deps): Upgrade the direct dependencies
chore(rtd): Group the tutorials by deployment model
chore(rtd): Rename Examples to Tutorials
chore: Deploy the default fallback CA bundle into the container image
chore: Remove obsolete YAML manifests
chore: Remove the post-install hook and static manifests
chore: Retire turncat
ci: Cross-compile the images, cache the layers
docs(api): Deprecate the v1 config API
docs(examples): Dogfood mediactl and the k8s:// URIs in the tutorials
docs(gateway): Add GatewayConfig loadBalancerServiceLabels
docs(gateway): Document L4 listeners and backend port rules
docs(gateway): Document the TURNRoute CRD
docs(gatewayclass): Document the Dataplane priorityClassName
docs(img): Redraw all figures
docs(sip): Describe separate SIP and media servers
docs: Document that IPv6 public-addrs are unbracketed
docs: Document TCPRoutes, dual-stack relays and TURN clusters
docs: Document the kernel offload and the session limits
docs: Replace the premium feature guide
docs: Rework the tutorials and proofread the docs
docs: Rewrite the docs for the media gateway
docs: Split the reference, add the TURN, SRT and RTSP guides
docs: Update the docs to the v2 operator rendering
test: Cover IPv6 listeners, relays and cluster routing
test: Port turncat tests to stunnerd
test: Randomize test ports for robustness

Gateway Operator

feat(api): Add the SIPTrunk type
feat(config): Patch the node address per dataplane pod
feat(dataplane): Add PriorityClassName to the Dataplane, fixes #77
feat(operator): Let builds add controllers to the Operator
feat(renderer): Add TURNRoute to route through an upstream TURN server
feat(renderer): Add loadBalancerServiceLabels to GatewayConfig
feat(renderer): Default the dataplane to 250m/128Mi, 1 CPU/512Mi
feat(renderer): Parse the SRT protocol
feat(renderer): Publish v2 dataplane configs
feat(renderer): Reject unsupported infrastructure parameters
feat(renderer): Render TCPRoutes and STUN mode in open source
feat(renderer): Render plain listener protocols as L4 servers
feat(renderer): Render the pqc-mode Gateway annotation
feat: Add TCPRoute support
feat: Export the build info to Prom metrics
feat: Run subsystems as manager runnables and gate the first render
feat: Watch Gateway API routes at the graduated v1 version
fix(config): Process the license update before the configs
fix(license): Make the default licence manager fail closed
fix(renderer): Set the IP family policy after the address
fix(updater): Own the DNS policy and IP family in the lens
fix: Format IPv6 CDS addresses properly, fixes #213
fix: Resolve node public address family-neutrally
fix: Watch TCPRoutes only where the cluster serves the CRD
fix: use net.JoinHostPort to handle ipv6 addresses
perf(cache): Bound the memory of the operator cache, fixes #78
refactor(api,store): Share the route spec across route kinds
refactor(controllers): Ask the REST mapper if a kind is served
refactor(renderer): Discover endpoints from EndpointSlices only
refactor(renderer): Render native v2 dataplane configs
refactor(renderer,lens): Unify the route pipeline over store.Route
refactor: Clean up operator before v2 release
refactor: Remove the legacy mode, v1alpha1 and v1alpha2
chore: Build on Go 1.27 and refresh dependencies
chore: Move the Gateway API test CRDs to v1.6.2
chore: Track the l7mp/stunner /v2 module path
ci: Build no artifacts in the public repo
ci: Cross-compile the images, cache the layers
test(config): Validate the configs from the CDS client
test(integration): Bump the test CRDs to Gateway API v1.6 and cover TCPRoute
test: Cover IPv6 public address rendering (LoadBalancer + NodePort)

Authentication service

feat(handler): Convert to take v2 dataplane configs
feat(handler): Hand out an ICE server per TURN auth policy
feat: Generate an ICE config for each public address
fix(handler): Validate the configs read from the CDS watch
fix: Track API changes in l7mp/stunner
refactor(handler): Use the TURN URIs of the API
chore: Track the l7mp/stunner /v2 module path and build on Go 1.27
ci: Cross-compile the images, cache the layers
test: Add test for IPv6 ICE server addresses
test: Port test auth handlers to pion/turn/v5