Connection MediaSoup

Connection: MediaSoup

An SFU. Every peer sends its media once to a mediasoup server, which forwards it to everyone else — so a call does not get more expensive for a client as peers are added.

var connection = connectionFactory.SelectConnection(ConnectionType.MediaSoup);

Nothing above the connection layer changes. The IConnection surface, joining and PeerResponse are on the Connection page; this page is about the client, the server, and the things that will bite you.

Read this first, because it is the single most misunderstood thing here:

The server is not part of WebRTCme

WebRTCme ships a client. The server it talks to is versatica's mediasoup-demo, which you deploy yourself.

That is a sharper statement than it sounds, because of what the client actually speaks. mediasoup itself has no wire protocol — it is a library that your own signalling server calls. The join, produce and newConsumer messages this client sends are the demo application's own signalling API, invented by that demo.

So "point it at a mediasoup server" is not enough. It has to be that server, and its API does change between versions: between 3.7.17 and 3.26 the response envelopes moved, ids were renamed, and SCTP negotiation disappeared — each of which broke this client.

The version is therefore pinned. WebRTCme.Connection/MediaSoup/server/Dockerfile pins the exact mediasoup-demo commit this client was verified against. Bump it deliberately and re-test; do not float it.

Verified against mediasoup 3.26.0, mediasoup-demo commit 558aadd82985b98e293564288ee95c69ccd05e98.

The server announces its version in a mediasoupVersion notification on connect, which is the quickest way to confirm what you are actually talking to.

What is in this repository

WebRTCme.Connection.MediaSoup	A C# port of mediasoup-client v3 — Device, Transport, Handler, Ortc, the SDP layer — plus the protoo client and the server contracts
MediaSoupConnection	Drives the port, and turns server events into the PeerResponse stream the middleware consumes
MediaSoup/server/	A recipe for running the server. Not a server

The port is the largest body of borrowed logic in the repository and the one most likely to drift from its JavaScript original, which is why Ortc, H264 and ScalabilityModes are the most heavily unit-tested things here — see Testing.

How a call is set up

sequenceDiagram
    autonumber
    participant C as client
    participant S as mediasoup-demo<br/>protoo over wss
    participant P as other peers

    C->>S: WebSocket connect — ?roomId=&peerId=
    S-->>C: notification: mediasoupVersion

    C->>S: getRouterRtpCapabilities
    S-->>C: router capabilities
    Note over C: Device.Load — works out what this<br/>client and the router can agree on

    C->>S: createWebRtcTransport (send)
    S-->>C: transport parameters — ICE, DTLS
    C->>S: createWebRtcTransport (recv)
    S-->>C: transport parameters

    C->>S: join(displayName, rtpCapabilities)
    S-->>C: peers already in the room

    C->>S: connectWebRtcTransport (DTLS)
    C->>S: produce(audio)
    C->>S: produce(video)
    Note over S,P: server forwards to every other peer

    S-->>C: newConsumer(peerId, producerId, ...)
    Note over C: consumer becomes a PeerResponse.PeerJoined
    S-->>C: notification: speakingPeers
    S-->>C: notification: peerClosed

Loading

Device.Load is where the capability negotiation happens — the part Ortc implements — and it is what makes a C# client able to talk to a server that has only ever spoken to browsers.

Running the server

mediasoup's worker is a native binary supported on Linux and macOS only; there is no Windows build. On a Windows machine, run it in Docker or on a Linux/macOS host.

cd WebRTCme.Connection/MediaSoup/server
docker build -t mediasoup-demo .

The build clones the pinned commit and compiles mediasoup's worker if no prebuilt binary exists for the platform, which takes a few minutes the first time.

Run it, announcing the address your clients will reach it on — not localhost, unless every client is on the same machine:

docker run -d --name mediasoup \
  -p 4443:4443/tcp \
  -p 44444:44444/udp -p 44444:44444/tcp \
  -e HTTP_LISTEN_PORT=4443 \
  -e NUM_WORKERS=1 \
  -e MEDIASOUP_LISTEN_IP=0.0.0.0 \
  -e MEDIASOUP_ANNOUNCED_ADDRESS=192.168.1.48 \
  -e DOMAIN=192.168.1.48 \
  -e DEBUG='mediasoup-demo-server:*' \
  mediasoup-demo

NUM_WORKERS is not a performance knob here — it decides how many ports you need. The server creates one worker per CPU by default and gives each its own RTC port, starting at 44444 and counting up. Thirty-two cores means ports 44444–44475. Publish every port you create, or media silently never flows while signalling looks perfectly healthy. One worker is plenty for testing.

MEDIASOUP_ANNOUNCED_ADDRESS is the address mediasoup puts in its ICE candidates, so it must be reachable from every client.

DOMAIN must be the address clients dial, and it defaults to localhost. The demo server rejects a WebSocket upgrade whose Origin does not match it, and it does so before any application logging - so a client on another machine fails like this, with an empty server log:

Connect: TcpSocketConnected
Connect: SecureSocketStreamConnected
Connect: SendingHandshakeToWebsocketServer
Cannot connect to the mediasoup server at wss://192.168.1.48:4443/?roomId=...:
  Websocket connection aborted unexpectedly. Check connection and socket security version/TLS version

Everything about that message points at TLS, and TLS is fine - it connected. docker logs says nothing at all, which is the tell: the rejection happened above the application. This cost an evening on 2026-09-17 with an iPhone that was otherwise working perfectly.

Check it is up. The log should end with Server started. Over HTTP the API is access controlled, so a 403 is the healthy answer — it means TLS terminated and the server replied:

curl -sk -o /dev/null -w '%{http_code}' https://192.168.1.48:4443/rooms/test   # 403

docker logs -f mediasoup shows every protoo request per peer, which is the fastest way to see how far a client got.

Pointing clients at it

MediaSoupServer:BaseUrl in the app's appsettings.json:

"MediaSoupServer": {
  "BaseUrl": "wss://192.168.1.48:4443",
  "Produce": true,
  "Consume": true,
  "UseDataChannel": true,
  "UseSimulcast": false,
  "ForceTcp": false,
  "ForceH264": false,
  "ForceVP9": false,
  "AudioOnly": false
}

Then join a room by name. Any number of clients using the same room name land in the same call.

Four things that will catch you

DOMAIN must match the host in the clients' BaseUrl, and there is only one of it. The server rejects a WebSocket whose HTTP Origin does not match its own, comparing scheme and host only — and a missing Origin counts as a mismatch and is refused with 403. A browser sets the header itself, from wherever the app is served; the mobile clients derive it from the server address. So a browser served from localhost and a phone reaching 192.168.1.48 cannot both be accepted at once. Serve the app on the same host the clients connect to, and set DOMAIN to that host.

Certificates. server/certs holds a self-signed pair. Browsers need it accepted once — visit https://<host>:4443/ and click through. Debug builds of the mobile clients fall back to a websocket that ignores certificate errors; release builds always validate, so a real deployment needs a real certificate.

A client killed abruptly does not disconnect. Swiping an app away sends no close frame, so the server keeps the peer until its own keepalive notices, which takes minutes. To test peers leaving, leave the call page instead.

The demo server runs a bot peer that opens a data consumer with no peer id attached. Client code assuming every consumer belongs to a peer will trip over it.

Simulcast, and why the demos turn it off

Simulcast — sending several resolutions at once and letting the server pick — is implemented and supported by the library. The demo apps set UseSimulcast: false deliberately, and the reason is worth knowing before you turn it on.

With simulcast on, this SFU's congestion control parks a native consumer on spatial layer 0 and flaps. Measured over one LAN: 64–159 kbit/s at 289x240 with simulcast, against 1781–1828 kbit/s at 578x480 with a single layer. The estimate only collapses when simulcast is in play, and probation stops with it.

That is mediasoup's congestion control rather than this client, which is why it is not fixed here. doc/KnownGaps.md has the measurements under "The SFU's bandwidth estimate collapses under simulcast".

Separately, Android cannot encode simulcast with this libwebrtc build at all.

LogServerStats: true asks the server for its own view of each consumer and of the receive transport, logged raw. Two extra requests per sample, so turn it on only while chasing something — it is what showed availableOutgoingBitrate collapsing to 47 kbit/s.

Voice activity comes free

The server runs an audio level observer and tells this client who is audible several times a second. It arrives as PeerResponseType.PeerMedia with MediaContext.Speaking.

The notification to use is speakingPeers, not activeSpeaker, and the names suggest the opposite of the truth:

• speakingPeers carries { peerVolumes: [ { peerId, volume } ] }, continuously, per peer. Membership of the list is itself the signal — the observer only reports producers above its threshold.
• activeSpeaker never fired once in testing: not for tones, not for continuous synthesised speech, not with a single audio producer left in the room.

Only differences between snapshots are reported, since these arrive several times a second.

Platform support

Blazor WebAssembly	works — verified
Android	works — verified
iOS	works — verified
Mac Catalyst	compile-verified only, never run
Windows	not verified

Blazor, Android and iOS have been verified together in one three-way call. See What works where for the whole picture.

Where next

Connection	IConnection, joining, PeerResponse
Connection: Signaling	The mesh alternative — simpler, and all five platforms
Demo apps	This path, working
mediasoup-demo	The server, upstream