Adding multiple streams to a single peer connection

[jc-chen]

Is there an easy way to add multiple video streams to a single peer connection?

Something like 'RTCPeerConnection.addTrack()' in the browser.

[ystreet]

Each input pad is a stream so you can have multiple input/output pads for multiple audio/video streams. The example already uses two streams, one for video and the other for audio.

[themanonthemoon1]

Okay so in the browser there would be a media source object with two different video tracks in this case. How would one go about having each track played in a separate video element?

Eventually I want to have separate video elements, but I still can't even get it working in a single video element. Even though I have both video tracks in a single video element, I can't find a way to 'switch' between each track. Both tracks are by default 'enabled', and if I try disabling the first track then I still don't see the second track (I just see a black video).

Is there any example code somewhere for dealing with multiple video tracks? Can't seem to find any anywhere.

[alexlapa]

@ystreet ,

The example already uses two streams, one for video and the other for audio.

Is there any way to send 2+ unrelated videos via single webrtcbin?

Each input pad is a stream so you can have multiple input/output pads for multiple audio/video streams.

I have tried naive calling add_video_source() twice (videotestsrc ! videoconvert ! vp8enc ! rtpvp8pay ! queue ! webrtcbin), which, i believe, is equivalent of peerConection.addTrack()/addStream(), changing add_video_source() to request additional pads:

    queue2
        .get_static_pad("src").unwrap()
        .link(&webrtcbin.get_request_pad("sink_%u").unwrap());

SDP seems to be correct:

v=0
o=- 4306522639330325592 0 IN IP4 0.0.0.0
s=-
t=0 0
a=ice-options:trickle
a=group:BUNDLE video0 video1
m=video 9 UDP/TLS/RTP/SAVPF 96
c=IN IP4 0.0.0.0
a=setup:actpass
a=ice-ufrag:KWyUvykHdcMZA2QAnHOB5Q7GB1I6ipLP
a=ice-pwd:VN1/BwXUNx0BSXfWuwSHs9gGqNy5mzyg
a=rtcp-mux
a=rtcp-rsize
a=sendrecv
a=rtpmap:96 VP8/90000
a=rtcp-fb:96 nack pli
a=framerate:30
a=ssrc:352868639 msid:user3161682870@host-6aeee50b webrtctransceiver0
a=ssrc:352868639 cname:user3161682870@host-6aeee50b
a=mid:video0
a=fingerprint:sha-256 2E:51:11:BA:F3:D5:D9:C8:79:2E:AC:F9:5A:F9:58:80:96:93:F3:F9:40:51:D3:B7:16:CD:EE:28:EC:79:8C:58
m=video 0 UDP/TLS/RTP/SAVPF 96
c=IN IP4 0.0.0.0
a=setup:actpass
a=ice-ufrag:KWyUvykHdcMZA2QAnHOB5Q7GB1I6ipLP
a=ice-pwd:VN1/BwXUNx0BSXfWuwSHs9gGqNy5mzyg
a=bundle-only
a=rtcp-mux
a=rtcp-rsize
a=sendrecv
a=rtpmap:96 VP8/90000
a=rtcp-fb:96 nack pli
a=framerate:30
a=ssrc:2097363443 msid:user3161682870@host-6aeee50b webrtctransceiver1
a=ssrc:2097363443 cname:user3161682870@host-6aeee50b
a=mid:video1
a=fingerprint:sha-256 2E:51:11:BA:F3:D5:D9:C8:79:2E:AC:F9:5A:F9:58:80:96:93:F3:F9:40:51:D3:B7:16:CD:EE:28:EC:79:8C:58

But onaddstream is fired only once (for video0). Related gst dot:

I have also tried adding multiple streams via add-transceiver, but couldn't find a way of connecting videosrc to transceivers sender. W3C spec states that tracks are supposed to be passed to peerConnection.addTransceiver(), which is not the case for webrtcbin's add-transceiver.

Trying to hack into webrtcbin by connecting video src's directly to rtpbin's send_rtp_sink_%u also failed, since no transceiver is being added(fails on this assert).

[ystreet]

This is a werbtcbin question and should be directed at the GStreamer mailing list or IRC channel.