Undefined RTCRtpListener behavior

[aboba]

Currently, it is not clear how the RTCRtpLIstener determines whether an RTP stream is configured to be processed by an existing RTCRtpReceiver object. Presumably this is based on the
RTCRtpReceiver.RTCRtpParameters.RTCRtpCodecParameters.payLoadType, RTCRtpReceiver.RTCRtpParameters.RTCRtpEncodingParameters.ssrc, and RTCRtpReceiver.RTCRtpParameters.receiverId attributes,
but how?

[robin-raymond]

Currently the RtcRtpReceiver object is setup to match incoming RTP streams based on 3 configurable options:

• payload type
• SSRC
• receiverId (aka appId)

Each one of these is "nullable" meaning, you can specify some combination of rules to match an incoming stream.

While this is simple, it can create some ambiguities which I will attempt to illustrate and suggest possible resolutions.

Use case (1): Specify an RtcRtpReceiver matching as:

• receiverId = "foo"
• payload type = 96

Then any stream which has a receiverId as "foo" and a payload of "96" will match that RtcRtpReceiver object. Any stream that does not match will be treated as "unhandled" and an "unhandled" event will fire.

Use case (2): Specify an RtcRtpReceiver matching as receiverId = "foo" but RTP stream drops "foo" receiverId header after first RTCP "ack" of that RTP stream

If a receiverId (DOMString) of "foo" comes in with an SSRC of "1000", as soon as the RTP stream gets acknowledged via an RTCP report then the receiverId can be dropped from the RTP packet extension header. But an implicit receiverId of "foo" must remain attached to every SSRC "1000" even though the receiverId is no longer present to continue allowing the RtcRtpReceiver matching rules to apply.

In my mind, the best way to handle this scenario.

If later suddenly another RTP SSRC stream of 1000 added a receiverId of "bar" instead of "foo", then that SSRC 1000 RTP stream no longer would match the "foo" rule and any further SSRC of 1000 without any receiverId would be considered as having a "bar" receiverId and not "foo".

Use case (3): Two RtcRtpReceiver objects matching as:
a) SSRC = 5000
b) SSRC = 5000, payload = 101

In this case, an RTP stream with 5000 and a payload ID that did not match 101 would match to RtcRtpReceiver object (a) and an RTP stream with SSRC 5000 but a payload ID of 101 would match to RtcRtpReceiver object (b). In other words, the most specific match wins over the general match.

Use case (4): Two RtcRtpReceiver objects matching as:
a) SSRC = 5000, payload = 96
b) SSRC = 5000, receiverId = "foo"

In this case, an RTP stream with SSRC 5000 and a payload ID of 96 would match (a). An RTP stream with SSRC 5000 and payload ID of 101 with no receiverId would not match (a) or (b) and would be "unhandled".

An RTP stream with SSRC 5000, payload ID of 96 and receiverId of "foo" would be ambiguous as both (a) or (b) could match. I think there needs to be precedence rules for matching in such cases. I'm open to debate as to what those rules should be. Would payload trump receiverId? Would it depend on RtcRtpReceiver construction order? Would it go to "unhandled" for the web app developer to resolve the conflict?

Use case (5): One RtcRtpReceiver object matching as:

• receiverId = "foo"

RTP stream (A) arrives with receiverId = "foo". RTP stream (B) arrives with receiverId = "foo".

Since there's only one "MediaStreamTrack" associated to the RtcRtpReceiver object, what happens to these two RTP streams? Yes, it's possible (perhaps likely) that one stream is replacing the other thus "last one wins" is an acceptable rule.

However, it's also possible this is not the case you have two streams and you need to decide what to do. Options:
i) last one wins, one RTP stream is thrown to the floor and the other is rendered
ii) both streams win. For audio, the system could auto-mix and treat each as contributing sources, but that's not an option for video so it would mean an event firing to give additional "MediaStreamTracks" as part of the RtcRtpReceiver which was not the original design intent for RtcRtpReceiver. The idea was that once "start(...)" was called on an RtcRtpReceiver, the MediaStreamTrack object would be valid to pre-wire to the rendered output to avoid any missing starting packets which might cause glitches.
iii) hybrid model - MediaStreamTrack is setup with "start(...)" to pre-wire, but an event fires to indicate a new MediaStreamTrack available but the existing MediaStreamTrack remains alive. This would imply some small buffering should be done to prevent glitches during the setup phase (which in my mind is already need for "unhandled" RTP streams so that's not a big deal for me).

Thoughts?

[martinthomson]

Don't make it so complex. Here's some simple rules.

1. SSRC wins.
2. Receiver ID is used in the absence of SSRC.
3. Payload type last, because that's trickier.

Once a match is made, the SSRC is copied in and then rule 1 is used from then on.

Mismatches are only a problem (and worth reporting perhaps ... but in something like stats) when:

a. Payload type for a known SSRC isn't known.
b. Receiver ID comes with with something else.

Both indicate either misconfiguration or SSRC collision between remote peers in the RTP session. Neither is of particular concern.

[robin-raymond]

The would presume that once an RtpReceiver is locked onto a stream, it's locked onto that stream forever and any new stream that comes in has to be fired as an unhandled event which needs a new RtpReceiver as oppose to allowing a receiver to be repurposed. In a conference scenario where streams are constantly changed, that means every change would need a new RtpReceiver per changed RTP stream (assuming it was coming from an SFU that didn't re-write the SSRC).

I'm okay with that, but we do still need to have exact rules.

[martinthomson]

See my response to Bernard. I think that you should be including the receiver ID header extension if you are forced to move to another SSRC. That would allow the new SSRC to be switched to if the SSRC had to change.

[aboba]

Suggestion from Peter:
http://lists.w3.org/Archives/Public/public-ortc/2014Sep/0011.html

1. If the SSRC is recognized, use it.
2. If the muxId is recognized, use it and latch the SSRC.
3. If the payload type uniquely routes the packets (only one RtpReceiver with that payload type), use it and latch the SSRC.
4. If none of those, fire the "unhandled" event.

[aboba]

Here is a strawman proposal:
http://lists.w3.org/Archives/Public/public-ortc/2014Dec/0004.html

[aboba]

A revised strawman:
http://lists.w3.org/Archives/Public/public-ortc/2014Dec/0007.html

[pthatcherg]

A few comments:

1. nit: withn -> within
2. Why are we using parameters.encodings[i].codecPayloadType on the
   receiver side? Shouldn't we be using parameters.codecs[i].payloadType? I
   would expect the value of parameters.encodings[i].codecPayloadType to
   basically be ignored on the receiver side (except perhaps for when needed
   for receiving FEC or RED)
3. I don't think we need to get to the detail of saying "ssrc is set to
   zero" or "muxId is set to the empty string". We just need to specify that
   with no SSRC set, the SSRC match is effectively a wild card, or likewise
   for the muxId.
4. If we get a packet with muxId X, and there is no RtpReceiver with muxId
   X, then it will route the packet to any RtpReceiver that understands the
   PT. I think this is a mistake. I think an unset muxId should not mean
   wild card. It should mean "match packets with an unset muxId".
5. As an alternative to the receiver table, I suggest a ruleset using just
   three simple maps: ssrc => receiver, muxId => receiver, and PT =>
   receiver. Then the rules would be:

a. (Fill ssrc => receiver) Set ssrc_table[ssrc] = receiver for each ssrc
in the parameters.encodings passed to each receiver by
RtpReceiver.receive().
b. (Fill muxid => receiver) Set muxid_table[muxId] = receiver for the
parameter.muxId passed to each receiver by Receiver.receive()
c. (Fill PT => receiver) Set pt_table[pt] = receiver for each pt in
paremeters.codecs passed to each receiver by Receiver.receiver(), but only
if parameters.muxId is unset and no ssrcs are in parameters.encodings.
d. (Avoid collisions) If ssrt_table[ssrc], muxid_table[muxId], or
pt_table[pt] would ever be set to a conflicting value, the call to
Receiver.receive (or Receiver.setTransport) will fail with an exception.

e. (SSRC match) If receiver = ssrc_table[packet.ssrc] is set, route the
packet to the receiver. But if the receiver does not understand packet.PT,
fire the unhandledrtp event.
f. (muxId match and SSRC latch) If packet.muxId is set and receiver =
muxid_table[packet.muxId] is set, route the packet to the receiver and set
ssrc_table[packet.ssrc] = receiver. But if the receiver does not
understand packet.PT, fire the unhandledrtp event.
g. (muxId unmatched) If packet.muxId is set and muxid_table[packet.muxId]
is unset, fire the unhandledrtp event.
d. (PT match and SSRC latch) If receiver = pt_table[packet.pt] is set,
route the packet to the receiver and set ssrc_table[packet.ssrc] = receiver.

Not that this means empty muxId is not wildcard, and receivers with set
muxIds and SSRCs also don't accept packets with muxIds or SSRCs outside of
the set ones. I think that's more what we want then so much wild card
behavior. And, I think this is a more simple set of rules that is easier
to implement.

What do you think?

On Tue, Dec 2, 2014 at 11:37 PM, aboba notifications@github.com wrote:

A revised strawman:
http://lists.w3.org/Archives/Public/public-ortc/2014Dec/0007.html

—
Reply to this email directly or view it on GitHub
#48 (comment).

[aboba]

Here are some thoughts from Peter:
http://lists.w3.org/Archives/Public/public-ortc/2014Dec/0018.html
And here are some potential use cases to evaluate against various proposals:
http://lists.w3.org/Archives/Public/public-ortc/2014Dec/0014.html

[aboba]

Peter Thatcher said:

“As an alternative to the receiver table, I suggest a ruleset using just three simple maps: ssrc => receiver, muxId => receiver, and PT => receiver. Then the rules would be:

a. (Fill ssrc => receiver) Set ssrc_table[ssrc] = receiver for each ssrc in the parameters.encodings passed to each receiver by RtpReceiver.receive().

[BA] OK.

b. (Fill muxid => receiver) Set muxid_table[muxId] = receiver for the parameter.muxId passed to each receiver by Receiver.receive()

[BA] OK.

c. (Fill PT => receiver) Set pt_table[pt] = receiver for each pt in parameters.codecs passed to each receiver by Receiver.receiver(), but only if parameters.muxId is unset and no ssrcs are in parameters.encodings.

[BA] Not sure what the "but only" means. Are we saying that if parameters.muxId is unset or there are ssrcs in parameters.encodings that we don't create a pt_table[pt] entry?

d. (Avoid collisions) If ssrt_table[ssrc], muxid_table[muxId], or pt_table[pt] would ever be set to a conflicting value, the call to Receiver.receive (or Receiver.setTransport) will fail with an exception.

[BA] OK.

e. (SSRC match) If receiver = ssrc_table[packet.ssrc] is set, route the packet to the receiver. But if the receiver does not understand packet.PT, fire the unhandledrtp event.

[BA] If we were allowed to set a pt_table[pt] entry in c), then the last sentence could be replaced by:
"But if pt_table[packet.PT] is unset or ssrc_table[packet.ssrc] !== pt_table[packet.PT] then fire the unhandledrtp event".

f. (muxId match and SSRC latch) If packet.muxId is set and receiver = muxid_table[packet.muxId] is set, route the packet to the receiver and set ssrc_table[packet.ssrc] = receiver. But if the receiver does not understand packet.PT, fire the unhandledrtp event.

[BA] If we were allowed to set a pt_table[pt] entry in c), then the last sentence could be replaced by:
"But if pt_table[packet.PT] is unset or muxid_table[packet.ssrc] !== pt_table[packet.PT] then fire the
unhandledrtp event".

g. (muxId unmatched) If packet.muxId is set and muxid_table[packet.muxId] is unset, fire the unhandledrtp event.

[BA] OK.

d. (PT match and SSRC latch) If receiver = pt_table[packet.pt] is set, route the packet to the receiver and set ssrc_table[packet.ssrc] = receiver.”

[BA] It is sometimes desirable to allow a sender to switch between sources, though it only sends one source at a time. In such a scenario we might route the first source to the receiver, then set ssrc_table[packet.ssrc] = receiver. When the sender switches sources, will the new source also be routed to the same receiver via the PT match rule, with another ssrc_table[packet.ssrc] = receiver entry being created, or will c) cause the PT match to no longer be operational, resulting in an unhandledrtp event?

[aboba]

Here a draft of proposed text for Section 8.3:

8.3 Matching rules

When the RTCRtpListener object receives an RTP packet over an RTCDtlsTransport, the RTCRtpListener attempts to determine which RTCRtpReceiver object to deliver the packet to, based on the values of the SSRC and payload type fields in the RTP header, as well as the value of the MID RTP header extension, if present.

The RTCRtpListener maintains four tables in order to facilitate matching: the ssrc_table which maps SSRC values to RTCRtpReceiver objects; the muxId_table which maps values of the MID header extension to RTCRtpReceiverobjects, the pt_table which maps payload type values to RTCRtpReceiver objects, and the receiver_table which keeps track of payload types supported by each receiver.

For an RTCRtpReceiver object receiver, table entries are added when receiver.receive(parameters) is called, and are removed when receiver.stop() is called. If receiver.receive(parameters) is called again, all entries referencing receiver are removed prior to adding new entries.

SSRC table: ssrc_table[parameters.encodings[i].ssrc] is set to receiver for each entry where parameters.encodings[i].ssrc is set, for values of i from 0 to the number of encodings. If ssrc_table[ssrc] is already set to a value other than receiver, then receiver.receive(parameters) will throw an InvalidParameters exception.

muxId table: If parameters.muxId is set, muxId_table[parameters.muxId] is set to receiver. If muxId_table[muxId] is already set to a value other than receiver, then receiver.receive(parameters) will throw an InvalidParameters exception.

payload type table: If parameters.muxId is unset and parameters.encodings[i].ssrc is unset for all values of i from 0 to the number of encodings, then add entries to pt_table by setting pt_table[parameters.codecs[j].payloadType] to receiver, for values of j from 0 to the number of codecs. If pt_table[pt] is already set to a value other than receiver, or parameters.codecs[j].payloadType is unset for any value of j from 0 to the number of codecs, then receiver.receive(parameters) will throw an InvalidParameters exception.

Receiver table: receiver_table[receiver] is set to an array of payload types, parameters.codecs[j].payloadtype, where j varies from 0 to the number of codecs.

When an RTP packet arrives, if ssrc_table[packet.ssrc] is set: set packet_receiver to ssrc_table[packet.ssrc] and check whether the value of packet.pt is included in any of the entries in receiver_table[packet_receiver]. If so, route the packet to packet_receiver. If packet.pt does not match, unhandledrtp event.

Else if packet.muxId is set: If muxId_table[packet.muxId] is unset, fire the unhandledrtp event, else set packet_receiver to muxId_table[packet.muxId] and check whether the value of packet.pt is included in any of the entries in receiver_table[packet_receiver]. If so, set ssrc_table[packet.ssrc] to packet_receiver and route the packet to packet_receiver. If packet.pt does not match, fire the unhandledrtp event.

Else if pt_table[packet.pt] is set: set packet_receiver to pt_table[packet.pt], set ssrc_table[packet.ssrc] to packet_receiver and route the packet to packet_receiver. Question: Do we also remove the pt_table[packet.pt] entry?

Else, fire the unhandledrtp event.

TODO: Revise this paragraph based on the outcome of the discussion on FEC/RTX/RED.

[pthatcherg]

On Mon, Jan 5, 2015 at 12:41 PM, aboba notifications@github.com wrote:

Peter Thatcher said:

“As an alternative to the receiver table, I suggest a ruleset using just
three simple maps: ssrc => receiver, muxId => receiver, and PT => receiver.
Then the rules would be:

a. (Fill ssrc => receiver) Set ssrc_table[ssrc] = receiver for each ssrc
in the parameters.encodings passed to each receiver by
RtpReceiver.receive().

[BA] OK.

b. (Fill muxid => receiver) Set muxid_table[muxId] = receiver for the
parameter.muxId passed to each receiver by Receiver.receive()

[BA] OK.

c. (Fill PT => receiver) Set pt_table[pt] = receiver for each pt in
parameters.codecs passed to each receiver by Receiver.receiver(), but only
if parameters.muxId is unset and no ssrcs are in parameters.encodings.

[BA] Not sure what the "but only" means. Are we saying that if
parameters.muxId is unset or there are ssrcs in parameters.encodings that
we don't create a pt_table[pt] entry?

​Yes, I'm suggesting that the JS can choose between demux by SSRC/muxID or
PT, but not both. Otherwise, the JS has to make sure PTs don't overlap,
even if the muxID used to demux. I'd prefer if different RtpSenders didn't
have to worry about PT overlap if they use the muxID or SSRCs to demux.
​

d. (Avoid collisions) If ssrt_table[ssrc], muxid_table[muxId], or

pt_table[pt] would ever be set to a conflicting value, the call to
Receiver.receive (or Receiver.setTransport) will fail with an exception.

[BA] OK.

e. (SSRC match) If receiver = ssrc_table[packet.ssrc] is set, route the
packet to the receiver. But if the receiver does not understand packet.PT,
fire the unhandledrtp event.

[BA] If we were allowed to set a pt_table[pt] entry in c), then the last
sentence could be replaced by:
"But if pt_table[packet.PT] is unset or ssrc_table[packet.ssrc] !==
pt_table[packet.PT] then fire the unhandledrtp event".

f. (muxId match and SSRC latch) If packet.muxId is set and receiver =
muxid_table[packet.muxId] is set, route the packet to the receiver and set
ssrc_table[packet.ssrc] = receiver. But if the receiver does not understand
packet.PT, fire the unhandledrtp event.

[BA] If we were allowed to set a pt_table[pt] entry in c), then the last
sentence could be replaced by:
"But if pt_table[packet.PT] is unset or muxid_table[packet.ssrc] !==
pt_table[packet.PT] then fire the
unhandledrtp event".

g. (muxId unmatched) If packet.muxId is set and muxid_table[packet.muxId]
is unset, fire the unhandledrtp event.

[BA] OK.

d. (PT match and SSRC latch) If receiver = pt_table[packet.pt] is set,
route the packet to the receiver and set ssrc_table[packet.ssrc] =
receiver.”

[BA] It is sometimes desirable to allow a sender to switch between
sources, though it only sends one source at a time. In such a scenario we
might route the first source to the receiver, then set
ssrc_table[packet.ssrc] = receiver. When the sender switches sources, will
the new source also be routed to the same receiver via the PT match rule,
with another ssrc_table[packet.ssrc] = receiver entry being created, or
will c) cause the PT match to no longer be operational, resulting in an
unhandledrtp event?

​I think you'd end up with another ssrc_table[packet.ssrc] entry. In other
words, if you have RtpReceiver r1 and pt_table = {101: r1} and yo
u
​receive​
packet p1 = {pt: 101, ssrc: 1001} and p2​ = {p1: 101, ssrc: 2002}, then
you'd end up with a
​ssrc_table = {1001: r1, 2002: r1}, and both packets would get routed to r1.

—
Reply to this email directly or view it on GitHub
#48 (comment).

[pthatcherg]

That sounds pretty good to me. The only thing I'm not sure about is
whether it's worth it to even mention the "receiver_table". Why not just
say "if packet.pt is in receiver.parameters.codec[n].payloadtype"? Why
bother with another table?

On Mon, Jan 12, 2015 at 12:12 PM, aboba notifications@github.com wrote:

Here a draft of proposed text for Section 8.3:

8.3 Matching rules

When the RTCRtpListener object receives an RTP packet over an
RTCDtlsTransport, the RTCRtpListener attempts to determine which
RTCRtpReceiver object to deliver the packet to, based on the values of the
SSRC and payload type fields in the RTP header, as well as the value of the
MID RTP header extension, if present.

The RTCRtpListener maintains four tables in order to facilitate matching:
the ssrc_table which maps SSRC values to RTCRtpReceiver objects; the
muxId_table which maps values of the MID header extension to
RTCRtpReceiverobjects, the pt_table which maps payload type values to
RTCRtpReceiver objects, and the receiver_table which keeps track of payload
types supported by each receiver.

For an RTCRtpReceiver object receiver, table entries are added when
receiver.receive(parameters) is called, and are removed when
receiver.stop() is called. If receiver.receive(parameters) is called again,
all entries referencing receiver are removed prior to adding new entries.

SSRC table: ssrc_table[parameters.encodings[i].ssrc] is set to receiver
for each entry where parameters.encodings[i].ssrc is set, for values of i
from 0 to the number of encodings. If ssrc_table[ssrc] is already set to a
value other than receiver, then receiver.receive(parameters) will throw an
InvalidParameters exception.

muxId table: If parameters.muxId is set, muxId_table[parameters.muxId] is
set to receiver. If muxId_table[muxId] is already set to a value other than
receiver, then receiver.receive(parameters) will throw an InvalidParameters
exception.

payload type table: If parameters.muxId is unset and
parameters.encodings[i].ssrc is unset for all values of i from 0 to the
number of encodings, then add entries to pt_table by setting
pt_table[parameters.codecs[j].payloadType] to receiver, for values of j
from 0 to the number of codecs. If pt_table[pt] is already set to a value
other than receiver, or parameters.codecs[j].payloadType is unset for any
value of j from 0 to the number of codecs, then
receiver.receive(parameters) will throw an InvalidParameters exception.

Receiver table: receiver_table[receiver] is set to an array of payload
types, parameters.codecs[j].payloadtype, where j varies from 0 to the
number of codecs.

When an RTP packet arrives, if ssrc_table[packet.ssrc] is set: set
packet_receiver to ssrc_table[packet.ssrc] and check whether the value of
packet.pt is included in any of the entries in
receiver_table[packet_receiver]. If so, route the packet to
packet_receiver. If packet.pt does not match, unhandledrtp event.

Else if packet.muxId is set: If muxId_table[packet.muxId] is unset, fire
the unhandledrtp event, else set packet_receiver to
muxId_table[packet.muxId] and check whether the value of packet.pt is
included in any of the entries in receiver_table[packet_receiver]. If so,
set ssrc_table[packet.ssrc] to packet_receiver and route the packet to
packet_receiver. If packet.pt does not match, fire the unhandledrtp event.

Else if pt_table[packet.pt] is set: set packet_receiver to pt_table[
packet.pt], set ssrc_table[packet.ssrc] to packet_receiver and route the
packet to packet_receiver. Question: Do we also remove the pt_table[
packet.pt] entry?

Else, fire the unhandledrtp event.

TODO: Revise this paragraph based on the outcome of the discussion on
FEC/RTX/RED.

—
Reply to this email directly or view it on GitHub
#48 (comment).