API is unable to handle inbound streams prior to arrival of answer

[dontcallmedom]

Originally reported on W3C Bugzilla ISSUE-21877 Tue, 30 Apr 2013 17:04:45 GMT
Reported by Martin Thomson
Assigned to

The ability to handle inbound media prior to an answer arriving is a key reason that the WebRTC API uses SDP offer/answer. However, this presents several challenges.

1. ICE: the offerer is expected to nominate pairs, but it cannot send connectivity checks prior to learning about the answerer's ufrag and password. Since the offerer is responsible for nominating, nothing happens until the answer arrives.

2a. Security Descriptions: the offerer doesn't know about the decryption keys for newly arrived streams.
2b. DTLS: the offerer can complete the handshake, but cannot render media until it obtains an identity assertion.

3a. Media: the offerer cannot correlate an inbound packet with an m= line by SSRC (if packet types are unique, then this could be used).
3b. The offerer does not know how to identify the inbound media stream and what grouping it fits within (CNAME, if present, is unsuitable because CNAME can be the same for multiple MediaStreams).

This has been discussed at some length in the IETF. It could be that the IETF arrive at some sort of conclusion on how to address this within SDP. I consider that unlikely (as unlikely as any statement of the form "... IETF arrive at some sort of conclusion ... SDP"). There are also discussions in the IETF around providing labeling on the media path.

It may be that some of these can't be addressed easily, or the situation changes depending on what multiplexing is in place, for example. Certainly, less of these factors are a problem when a new stream is added to a multiplexed bundle on an existing transport (only 3a, 3b and maybe 2a apply).

Regardless of what the IETF concludes it's likely that there will be a need for API changes to support this feature (3b in particular).

CU-RTC-Web addresses this problem by providing an event for an as-yet-unsignaled media flow. This event indicates SSRC and packet type of the unknown packet. It also requests that browsers buffer this inbound stream for a short time. Buffering would probably just include enough time to allow the application to process and respond to the event, there is no need to allow enough time to perform any signaling, which would undermine the whole "real time" aspect of this and potentially cause buffering space to be larger than is tolerable.

The CU-RTC-Web solution is fairly simple because it pushes responsibility onto the application. That might be an undesirable property, though it might be argued that this is an advanced usage of the API and the extra application responsibility is commensurate with that.

An alternative solution involves the creation of "dummy" media streams that exist for a short time. There are several variations on this. In one solution the streams that are created are single track streams with undefined identifiers. Another alternative suggests that a single default stream is created with all unannounced tracks attached to it.

There is another alternative to be considered for 3b, which is that identifiers could be allocated locally (CU-RTC-Web does this). In order to correlate streams with the ones on the remote peer a new attribute, remoteid, is added to include the identifier used by the source. The 'remoteid' would only be populated once signaling arrived.

'remoteid' provides a measure of stability that the "dummy" stream configurations cannot provide. 'remoteid' would probably supplant the recently added 'remote' attribute.