unable to get remote stream in case when a “remote stream” is attached by calling “peer.addStream(remoteStream)

[muaz-khan]

Issue: unable to get remote stream in case when a “remote stream” is attached by calling “peer.addStream ( remoteStream )”

Is it beneficial to attach remote stream?

Yeah, it allows us overcome burden from a single peer. Burden will be shared.

Also, we can support a wide range of peer’s connectivity.

How to attach remote stream?

var MediaStream = window.webkitMediaStream || window.MediaStream;

firstPeer.onaddstream = function(remoteSteam) {
      remoteStream = new MediaStream(remoteSteam.audioTracks, remoteSteam.videoTracks);
      otherPeer.addStream(remoteStream);  /* attaching remote stream */
};

What I want to do?

1. First peer will handle first three peers
2. 2nd peer will handle next three peers (5, 6, 7)
3. 5th peer will handle next three peers (8,9,10)
4. 8th peer will handle next three peers (11, 12, 13)
5. And so on.

We will get following benefits (in case of success):

1. Video will never freeze for 11th and upper peers
2. 1st peer don’t need to handle all participants
3. You don’t need to install/buy a middle server (like Asterisk) for small projects (to support a few hundred peers’ connectivity)

A known bug in chromium: peer.onicecandidate not fires for 11th peer. You don’t need to worry about that bug!!

Pitfall: If peer number 1, 2, 5, 8 … tries to leave the room without informing other peers to play a host role.

If peer number 2 leaves the room, without informing master peer or peer number 5 to handling upcoming peers.

A demo experiment to test it:

https://googledrive.com/host/0B6GWd_dUUTT8V1Fodm9WQldkb28/

Open 4 tabs....1st table should create room……..join room from other tabs.....see the behavior of the 4th tab.

[muaz-khan]

unable to generate ice candidates (on the answerer side) when attaching remote stream..

a=candidate:2437072876 1 udp 2113937151 192.168.1.2 54082 typ host generation 0
a=candidate:2437072876 2 udp 2113937151 192.168.1.2 54082 typ host generation 0
a=candidate:2437072876 1 udp 2113937151 192.168.1.2 54082 typ host generation 0
a=candidate:2437072876 2 udp 2113937151 192.168.1.2 54082 typ host generation 0
a=candidate:941443129 1 udp 1845501695 39.47.69.177 10136 typ srflx raddr 192.168.1.2 rport 54082 generation 0
a=candidate:941443129 2 udp 1845501695 39.47.69.177 10136 typ srflx raddr 192.168.1.2 rport 54082 generation 0
a=candidate:941443129 1 udp 1845501695 39.47.69.177 10136 typ srflx raddr 192.168.1.2 rport 54082 generation 0
a=candidate:941443129 2 udp 1845501695 39.47.69.177 10136 typ srflx raddr 192.168.1.2 rport 54082 generation 0

--------offer sdp provided by offerer

v=0
o=- 2216700829 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE audio video
a=msid-semantic: WMS
m=audio 1 RTP/SAVPF 103 104 111 0 8 107 106 105 13 126
c=IN IP4 0.0.0.0
a=rtcp:1 IN IP4 0.0.0.0
a=ice-ufrag:Yy0jsw+XcU7Ovmw4
a=ice-pwd:9kjBUxNEoOOO0yViUNEH9Yhe
a=ice-options:google-ice
a=fingerprint:sha-256 ED:B9:CF:19:5F:A3:F4:D3:31:FC:F5:18:3C:AF:60:9E:B1:52:A5:32:69:8A:22:63:05:AD:98:93:ED:33:2A:7E
a=recvonly
a=mid:audio
a=rtcp-mux
a=crypto:1 AES_CM_128_HMAC_SHA1_80 inline:CFkD+WaozWTFg41l4mi5rdoAscTYMJa8CB1HgVpW
a=rtpmap:103 ISAC/16000
a=rtpmap:104 ISAC/32000
a=rtpmap:111 opus/48000/2
a=fmtp:111 minptime=10
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:107 CN/48000
a=rtpmap:106 CN/32000
a=rtpmap:105 CN/16000
a=rtpmap:13 CN/8000
a=rtpmap:126 telephone-event/8000
a=maxptime:60
m=video 1 RTP/SAVPF 100 116 117
c=IN IP4 0.0.0.0
a=rtcp:1 IN IP4 0.0.0.0
a=ice-ufrag:Yy0jsw+XcU7Ovmw4
a=ice-pwd:9kjBUxNEoOOO0yViUNEH9Yhe
a=ice-options:google-ice
a=fingerprint:sha-256 ED:B9:CF:19:5F:A3:F4:D3:31:FC:F5:18:3C:AF:60:9E:B1:52:A5:32:69:8A:22:63:05:AD:98:93:ED:33:2A:7E
a=recvonly
a=mid:video
a=rtcp-mux
a=crypto:1 AES_CM_128_HMAC_SHA1_80 inline:CFkD+WaozWTFg41l4mi5rdoAscTYMJa8CB1HgVpW
a=rtpmap:100 VP8/90000
a=rtpmap:116 red/90000
a=rtpmap:117 ulpfec/90000
RTCPeerConnection-v1.4.js:191

--------answer

sdp: v=0
o=- 315182876 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE audio video
a=msid-semantic: WMS
m=audio 1 RTP/SAVPF 103 104 111 0 8 107 106 105 13 126
a=crypto:1 AES_CM_128_HMAC_SHA1_80 inline:Do0n1o1bjqhwgfo7jsfqBi2pgBx9i9egkjngrjkf
c=IN IP4 0.0.0.0
a=rtcp:1 IN IP4 0.0.0.0
a=ice-ufrag:ZQglAeIL9/0MtR+R
a=ice-pwd:gbAu8knZaPhZrBQc9mDjK2AT
a=fingerprint:sha-256 50:DD:32:5B:31:92:C0:CF:77:2A:DC:66:45:29:6C:58:73:A8:77:AF:10:38:D0:FA:53:D9:55:26:0A:DC:B2:6A
a=recvonly
a=mid:audio
a=rtcp-mux
a=rtpmap:103 ISAC/16000
a=rtpmap:104 ISAC/32000
a=rtpmap:111 opus/48000/2
a=fmtp:111 minptime=10
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:107 CN/48000
a=rtpmap:106 CN/32000
a=rtpmap:105 CN/16000
a=rtpmap:13 CN/8000
a=rtpmap:126 telephone-event/8000
a=maxptime:60
m=video 1 RTP/SAVPF 100 116 117
a=crypto:1 AES_CM_128_HMAC_SHA1_80 inline:Do0n1o1bjqhwgfo7jsfqBi2pgBx9i9egkjngrjkf
c=IN IP4 0.0.0.0
a=rtcp:1 IN IP4 0.0.0.0
a=ice-ufrag:ZQglAeIL9/0MtR+R
a=ice-pwd:gbAu8knZaPhZrBQc9mDjK2AT
a=fingerprint:sha-256 50:DD:32:5B:31:92:C0:CF:77:2A:DC:66:45:29:6C:58:73:A8:77:AF:10:38:D0:FA:53:D9:55:26:0A:DC:B2:6A
a=recvonly
a=mid:video
a=rtcp-mux
a=rtpmap:100 VP8/90000
a=rtpmap:116 red/90000
a=rtpmap:117 ulpfec/90000

Maybe RTP ports already opened....We may need to change them dynamically.

[muaz-khan]

Latest one-page demo on attaching remote media streams. Still fails.

[peili]

Any process on this issue so far?

[muaz-khan]

This issue is not fixed yet. Attachment of remote media streams still failing on both chrome and firefox. I tried audio-only remote streams too but no success.

[peili]

Do you know when Google/Mozilla are going to support this feature? I couldn't find clear informations on the roadmaps..

[muaz-khan]

I'm not sure, however it seems that audio-only remote streams' attachment works fine.

[muaz-khan]

A general comment is that we only support one audio source today in Chrome given that we only have one ADM. To me it sounds like we need a new type of source here where the audio is driven by the remote peer. Perhaps it would be less complicated to build something initially if we only worked in relay mode (one source).

Reference: https://code.google.com/p/webrtc/issues/detail?id=383

.....take a remote stream from one peer connection and adding it to another peer connection and having it Just Work™ for that second peer.

Implementation awaited!

[RonMen]

Hi @muaz-khan,

I think this is what I thought about on my other issue questioning. The use case is:

1.) Peer1 connects with audio only
2.) Peer2 connects with audio only
... later on one/both switch
3.) Peer1/Peer2 switches over to remove audio only stream and instead add a video+audio stream

In this case I get a black video on the peer this stream is delivered to sending a "new" offer and there are removeStream + addStream events fired up but also the data of the newly added stream isn't send.

I tried multiple settings and also tried to find an answer but it doesn't seem to work. Also the sample implementation presented in the webrtc issue tracker isn't working like expected: https://code.google.com/p/webrtc/issues/detail?id=1086

Do you have any idea how to bring this up and running or is there something missing in browsers webrtc implementation for now?

[muaz-khan]

Try this demo: Switching streams from screen-sharing to audio+video at runtime. (Renegotiation)

1. Both peers are sharing screen
2. You can switch to audio+video stream; old stream (i.e. screen-sharing) will be removed.

Don't need to add ice candidates when renegotiating.

[RonMen]

Dear @muaz-khan, thx for your sample. I also found something else since setting the localDescription failed while switching like explained here: https://groups.google.com/forum/#!msg/discuss-webrtc/ets-eex6H-0/bkN2UwZI-gEJ

I changed the {optional: [{DtlsSrtpKeyAgreement: true}]} of the peerconnection and I tried without it and the streams are switched well now.

[skanagavelu]

We need this idea. This communication is low cost for end user.
Mauz; How this is supported for screen share?

[skanagavelu]

Hi Muaz,

In your example below

firstPeer.onaddstream = function(remoteSteam) {
remoteStream = new MediaStream(remoteSteam.audioTracks, remoteSteam.videoTracks);
otherPeer.addStream(remoteStream); /* attaching remote stream */
};

Why we are recreating remoteStream again using new MediaStream ?

And pls tell me what we have to do for screen share while creating new instance of MediaStream...?

[muaz-khan]

There are some bugs preventing video tracks to be forwarded. In screen sharing apps; we need to retransmit video RTP packets; which seems failing on both chrome and firefox.

In the example; new media stream is created; because in old days, chrome were throwing not-supported exceptions for MediaStream because only LocalMediaStream objects were permitted to be attached using peer.addStream method.

Nowadays, you can attach remote stream directly, on chrome; without initiating new object; because both local and remote streams have identical MediaStream type.

I think identical ports (RTP or UDP) are causing failures to attach remote media streams. Though, I don't know the exact issue. Maybe ICE Agent is unable to locate right ports to retransmit remote media stream.

Maximum peer connections limit is 256; so it is preferred to use application specific bandwidth parameters to stream low-quality packets; also specify width/height constraints to handle freezing issues.

[skanagavelu]

FYI : https://groups.google.com/forum/#!topic/discuss-webrtc/CnEHFxjxE_8

[peili]

Any updates on "attaching remote audio/video stream"?
Audio seems to work. Video not?