Undefined RTCRtpListener behavior

[aboba]

Currently, it is not clear how the RTCRtpLIstener determines whether an RTP stream is configured to be processed by an existing RTCRtpReceiver object. Presumably this is based on the
RTCRtpReceiver.RTCRtpParameters.RTCRtpCodecParameters.payLoadType, RTCRtpReceiver.RTCRtpParameters.RTCRtpEncodingParameters.ssrc, and RTCRtpReceiver.RTCRtpParameters.receiverId attributes,
but how?

[robin-raymond]

Currently the RtcRtpReceiver object is setup to match incoming RTP streams based on 3 configurable options:

• payload type
• SSRC
• receiverId (aka appId)

Each one of these is "nullable" meaning, you can specify some combination of rules to match an incoming stream.

While this is simple, it can create some ambiguities which I will attempt to illustrate and suggest possible resolutions.

Use case (1): Specify an RtcRtpReceiver matching as:

• receiverId = "foo"
• payload type = 96

Then any stream which has a receiverId as "foo" and a payload of "96" will match that RtcRtpReceiver object. Any stream that does not match will be treated as "unhandled" and an "unhandled" event will fire.

Use case (2): Specify an RtcRtpReceiver matching as receiverId = "foo" but RTP stream drops "foo" receiverId header after first RTCP "ack" of that RTP stream

If a receiverId (DOMString) of "foo" comes in with an SSRC of "1000", as soon as the RTP stream gets acknowledged via an RTCP report then the receiverId can be dropped from the RTP packet extension header. But an implicit receiverId of "foo" must remain attached to every SSRC "1000" even though the receiverId is no longer present to continue allowing the RtcRtpReceiver matching rules to apply.

In my mind, the best way to handle this scenario.

If later suddenly another RTP SSRC stream of 1000 added a receiverId of "bar" instead of "foo", then that SSRC 1000 RTP stream no longer would match the "foo" rule and any further SSRC of 1000 without any receiverId would be considered as having a "bar" receiverId and not "foo".

Use case (3): Two RtcRtpReceiver objects matching as:
a) SSRC = 5000
b) SSRC = 5000, payload = 101

In this case, an RTP stream with 5000 and a payload ID that did not match 101 would match to RtcRtpReceiver object (a) and an RTP stream with SSRC 5000 but a payload ID of 101 would match to RtcRtpReceiver object (b). In other words, the most specific match wins over the general match.

Use case (4): Two RtcRtpReceiver objects matching as:
a) SSRC = 5000, payload = 96
b) SSRC = 5000, receiverId = "foo"

In this case, an RTP stream with SSRC 5000 and a payload ID of 96 would match (a). An RTP stream with SSRC 5000 and payload ID of 101 with no receiverId would not match (a) or (b) and would be "unhandled".

An RTP stream with SSRC 5000, payload ID of 96 and receiverId of "foo" would be ambiguous as both (a) or (b) could match. I think there needs to be precedence rules for matching in such cases. I'm open to debate as to what those rules should be. Would payload trump receiverId? Would it depend on RtcRtpReceiver construction order? Would it go to "unhandled" for the web app developer to resolve the conflict?

Use case (5): One RtcRtpReceiver object matching as:

• receiverId = "foo"

RTP stream (A) arrives with receiverId = "foo". RTP stream (B) arrives with receiverId = "foo".

Since there's only one "MediaStreamTrack" associated to the RtcRtpReceiver object, what happens to these two RTP streams? Yes, it's possible (perhaps likely) that one stream is replacing the other thus "last one wins" is an acceptable rule.

However, it's also possible this is not the case you have two streams and you need to decide what to do. Options:
i) last one wins, one RTP stream is thrown to the floor and the other is rendered
ii) both streams win. For audio, the system could auto-mix and treat each as contributing sources, but that's not an option for video so it would mean an event firing to give additional "MediaStreamTracks" as part of the RtcRtpReceiver which was not the original design intent for RtcRtpReceiver. The idea was that once "start(...)" was called on an RtcRtpReceiver, the MediaStreamTrack object would be valid to pre-wire to the rendered output to avoid any missing starting packets which might cause glitches.
iii) hybrid model - MediaStreamTrack is setup with "start(...)" to pre-wire, but an event fires to indicate a new MediaStreamTrack available but the existing MediaStreamTrack remains alive. This would imply some small buffering should be done to prevent glitches during the setup phase (which in my mind is already need for "unhandled" RTP streams so that's not a big deal for me).

Thoughts?

[martinthomson]

Don't make it so complex. Here's some simple rules.

1. SSRC wins.
2. Receiver ID is used in the absence of SSRC.
3. Payload type last, because that's trickier.

Once a match is made, the SSRC is copied in and then rule 1 is used from then on.

Mismatches are only a problem (and worth reporting perhaps ... but in something like stats) when:

a. Payload type for a known SSRC isn't known.
b. Receiver ID comes with with something else.

Both indicate either misconfiguration or SSRC collision between remote peers in the RTP session. Neither is of particular concern.

[robin-raymond]

The would presume that once an RtpReceiver is locked onto a stream, it's locked onto that stream forever and any new stream that comes in has to be fired as an unhandled event which needs a new RtpReceiver as oppose to allowing a receiver to be repurposed. In a conference scenario where streams are constantly changed, that means every change would need a new RtpReceiver per changed RTP stream (assuming it was coming from an SFU that didn't re-write the SSRC).

I'm okay with that, but we do still need to have exact rules.

[martinthomson]

See my response to Bernard. I think that you should be including the receiver ID header extension if you are forced to move to another SSRC. That would allow the new SSRC to be switched to if the SSRC had to change.

[aboba]

Suggestion from Peter:
http://lists.w3.org/Archives/Public/public-ortc/2014Sep/0011.html

1. If the SSRC is recognized, use it.
2. If the muxId is recognized, use it and latch the SSRC.
3. If the payload type uniquely routes the packets (only one RtpReceiver with that payload type), use it and latch the SSRC.
4. If none of those, fire the "unhandled" event.

[aboba]

Here is a strawman proposal:
http://lists.w3.org/Archives/Public/public-ortc/2014Dec/0004.html

[aboba]

A revised strawman:
http://lists.w3.org/Archives/Public/public-ortc/2014Dec/0007.html

[pthatcherg]

A few comments:

1. nit: withn -> within
2. Why are we using parameters.encodings[i].codecPayloadType on the
   receiver side? Shouldn't we be using parameters.codecs[i].payloadType? I
   would expect the value of parameters.encodings[i].codecPayloadType to
   basically be ignored on the receiver side (except perhaps for when needed
   for receiving FEC or RED)
3. I don't think we need to get to the detail of saying "ssrc is set to
   zero" or "muxId is set to the empty string". We just need to specify that
   with no SSRC set, the SSRC match is effectively a wild card, or likewise
   for the muxId.
4. If we get a packet with muxId X, and there is no RtpReceiver with muxId
   X, then it will route the packet to any RtpReceiver that understands the
   PT. I think this is a mistake. I think an unset muxId should not mean
   wild card. It should mean "match packets with an unset muxId".
5. As an alternative to the receiver table, I suggest a ruleset using just
   three simple maps: ssrc => receiver, muxId => receiver, and PT =>
   receiver. Then the rules would be:

a. (Fill ssrc => receiver) Set ssrc_table[ssrc] = receiver for each ssrc
in the parameters.encodings passed to each receiver by
RtpReceiver.receive().
b. (Fill muxid => receiver) Set muxid_table[muxId] = receiver for the
parameter.muxId passed to each receiver by Receiver.receive()
c. (Fill PT => receiver) Set pt_table[pt] = receiver for each pt in
paremeters.codecs passed to each receiver by Receiver.receiver(), but only
if parameters.muxId is unset and no ssrcs are in parameters.encodings.
d. (Avoid collisions) If ssrt_table[ssrc], muxid_table[muxId], or
pt_table[pt] would ever be set to a conflicting value, the call to
Receiver.receive (or Receiver.setTransport) will fail with an exception.

e. (SSRC match) If receiver = ssrc_table[packet.ssrc] is set, route the
packet to the receiver. But if the receiver does not understand packet.PT,
fire the unhandledrtp event.
f. (muxId match and SSRC latch) If packet.muxId is set and receiver =
muxid_table[packet.muxId] is set, route the packet to the receiver and set
ssrc_table[packet.ssrc] = receiver. But if the receiver does not
understand packet.PT, fire the unhandledrtp event.
g. (muxId unmatched) If packet.muxId is set and muxid_table[packet.muxId]
is unset, fire the unhandledrtp event.
d. (PT match and SSRC latch) If receiver = pt_table[packet.pt] is set,
route the packet to the receiver and set ssrc_table[packet.ssrc] = receiver.

Not that this means empty muxId is not wildcard, and receivers with set
muxIds and SSRCs also don't accept packets with muxIds or SSRCs outside of
the set ones. I think that's more what we want then so much wild card
behavior. And, I think this is a more simple set of rules that is easier
to implement.

What do you think?

On Tue, Dec 2, 2014 at 11:37 PM, aboba notifications@github.com wrote:

A revised strawman:
http://lists.w3.org/Archives/Public/public-ortc/2014Dec/0007.html

—
Reply to this email directly or view it on GitHub
#48 (comment).

[aboba]

Here are some thoughts from Peter:
http://lists.w3.org/Archives/Public/public-ortc/2014Dec/0018.html
And here are some potential use cases to evaluate against various proposals:
http://lists.w3.org/Archives/Public/public-ortc/2014Dec/0014.html

[aboba]

Peter Thatcher said:

“As an alternative to the receiver table, I suggest a ruleset using just three simple maps: ssrc => receiver, muxId => receiver, and PT => receiver. Then the rules would be:

a. (Fill ssrc => receiver) Set ssrc_table[ssrc] = receiver for each ssrc in the parameters.encodings passed to each receiver by RtpReceiver.receive().

[BA] OK.

b. (Fill muxid => receiver) Set muxid_table[muxId] = receiver for the parameter.muxId passed to each receiver by Receiver.receive()

[BA] OK.

c. (Fill PT => receiver) Set pt_table[pt] = receiver for each pt in parameters.codecs passed to each receiver by Receiver.receiver(), but only if parameters.muxId is unset and no ssrcs are in parameters.encodings.

[BA] Not sure what the "but only" means. Are we saying that if parameters.muxId is unset or there are ssrcs in parameters.encodings that we don't create a pt_table[pt] entry?

d. (Avoid collisions) If ssrt_table[ssrc], muxid_table[muxId], or pt_table[pt] would ever be set to a conflicting value, the call to Receiver.receive (or Receiver.setTransport) will fail with an exception.

[BA] OK.

e. (SSRC match) If receiver = ssrc_table[packet.ssrc] is set, route the packet to the receiver. But if the receiver does not understand packet.PT, fire the unhandledrtp event.

[BA] If we were allowed to set a pt_table[pt] entry in c), then the last sentence could be replaced by:
"But if pt_table[packet.PT] is unset or ssrc_table[packet.ssrc] !== pt_table[packet.PT] then fire the unhandledrtp event".

f. (muxId match and SSRC latch) If packet.muxId is set and receiver = muxid_table[packet.muxId] is set, route the packet to the receiver and set ssrc_table[packet.ssrc] = receiver. But if the receiver does not understand packet.PT, fire the unhandledrtp event.

[BA] If we were allowed to set a pt_table[pt] entry in c), then the last sentence could be replaced by:
"But if pt_table[packet.PT] is unset or muxid_table[packet.ssrc] !== pt_table[packet.PT] then fire the
unhandledrtp event".

g. (muxId unmatched) If packet.muxId is set and muxid_table[packet.muxId] is unset, fire the unhandledrtp event.

[BA] OK.

d. (PT match and SSRC latch) If receiver = pt_table[packet.pt] is set, route the packet to the receiver and set ssrc_table[packet.ssrc] = receiver.”

[BA] It is sometimes desirable to allow a sender to switch between sources, though it only sends one source at a time. In such a scenario we might route the first source to the receiver, then set ssrc_table[packet.ssrc] = receiver. When the sender switches sources, will the new source also be routed to the same receiver via the PT match rule, with another ssrc_table[packet.ssrc] = receiver entry being created, or will c) cause the PT match to no longer be operational, resulting in an unhandledrtp event?

[aboba]

Here a draft of proposed text for Section 8.3:

8.3 Matching rules

When the RTCRtpListener object receives an RTP packet over an RTCDtlsTransport, the RTCRtpListener attempts to determine which RTCRtpReceiver object to deliver the packet to, based on the values of the SSRC and payload type fields in the RTP header, as well as the value of the MID RTP header extension, if present.

The RTCRtpListener maintains four tables in order to facilitate matching: the ssrc_table which maps SSRC values to RTCRtpReceiver objects; the muxId_table which maps values of the MID header extension to RTCRtpReceiverobjects, the pt_table which maps payload type values to RTCRtpReceiver objects, and the receiver_table which keeps track of payload types supported by each receiver.

For an RTCRtpReceiver object receiver, table entries are added when receiver.receive(parameters) is called, and are removed when receiver.stop() is called. If receiver.receive(parameters) is called again, all entries referencing receiver are removed prior to adding new entries.

SSRC table: ssrc_table[parameters.encodings[i].ssrc] is set to receiver for each entry where parameters.encodings[i].ssrc is set, for values of i from 0 to the number of encodings. If ssrc_table[ssrc] is already set to a value other than receiver, then receiver.receive(parameters) will throw an InvalidParameters exception.

muxId table: If parameters.muxId is set, muxId_table[parameters.muxId] is set to receiver. If muxId_table[muxId] is already set to a value other than receiver, then receiver.receive(parameters) will throw an InvalidParameters exception.

payload type table: If parameters.muxId is unset and parameters.encodings[i].ssrc is unset for all values of i from 0 to the number of encodings, then add entries to pt_table by setting pt_table[parameters.codecs[j].payloadType] to receiver, for values of j from 0 to the number of codecs. If pt_table[pt] is already set to a value other than receiver, or parameters.codecs[j].payloadType is unset for any value of j from 0 to the number of codecs, then receiver.receive(parameters) will throw an InvalidParameters exception.

Receiver table: receiver_table[receiver] is set to an array of payload types, parameters.codecs[j].payloadtype, where j varies from 0 to the number of codecs.

When an RTP packet arrives, if ssrc_table[packet.ssrc] is set: set packet_receiver to ssrc_table[packet.ssrc] and check whether the value of packet.pt is included in any of the entries in receiver_table[packet_receiver]. If so, route the packet to packet_receiver. If packet.pt does not match, unhandledrtp event.

Else if packet.muxId is set: If muxId_table[packet.muxId] is unset, fire the unhandledrtp event, else set packet_receiver to muxId_table[packet.muxId] and check whether the value of packet.pt is included in any of the entries in receiver_table[packet_receiver]. If so, set ssrc_table[packet.ssrc] to packet_receiver and route the packet to packet_receiver. If packet.pt does not match, fire the unhandledrtp event.

Else if pt_table[packet.pt] is set: set packet_receiver to pt_table[packet.pt], set ssrc_table[packet.ssrc] to packet_receiver and route the packet to packet_receiver. Question: Do we also remove the pt_table[packet.pt] entry?

Else, fire the unhandledrtp event.

TODO: Revise this paragraph based on the outcome of the discussion on FEC/RTX/RED.

[pthatcherg]

On Mon, Jan 5, 2015 at 12:41 PM, aboba notifications@github.com wrote:

Peter Thatcher said:

“As an alternative to the receiver table, I suggest a ruleset using just
three simple maps: ssrc => receiver, muxId => receiver, and PT => receiver.
Then the rules would be:

a. (Fill ssrc => receiver) Set ssrc_table[ssrc] = receiver for each ssrc
in the parameters.encodings passed to each receiver by
RtpReceiver.receive().

[BA] OK.

b. (Fill muxid => receiver) Set muxid_table[muxId] = receiver for the
parameter.muxId passed to each receiver by Receiver.receive()

[BA] OK.

c. (Fill PT => receiver) Set pt_table[pt] = receiver for each pt in
parameters.codecs passed to each receiver by Receiver.receiver(), but only
if parameters.muxId is unset and no ssrcs are in parameters.encodings.

[BA] Not sure what the "but only" means. Are we saying that if
parameters.muxId is unset or there are ssrcs in parameters.encodings that
we don't create a pt_table[pt] entry?

​Yes, I'm suggesting that the JS can choose between demux by SSRC/muxID or
PT, but not both. Otherwise, the JS has to make sure PTs don't overlap,
even if the muxID used to demux. I'd prefer if different RtpSenders didn't
have to worry about PT overlap if they use the muxID or SSRCs to demux.
​

d. (Avoid collisions) If ssrt_table[ssrc], muxid_table[muxId], or

pt_table[pt] would ever be set to a conflicting value, the call to
Receiver.receive (or Receiver.setTransport) will fail with an exception.

[BA] OK.

e. (SSRC match) If receiver = ssrc_table[packet.ssrc] is set, route the
packet to the receiver. But if the receiver does not understand packet.PT,
fire the unhandledrtp event.

[BA] If we were allowed to set a pt_table[pt] entry in c), then the last
sentence could be replaced by:
"But if pt_table[packet.PT] is unset or ssrc_table[packet.ssrc] !==
pt_table[packet.PT] then fire the unhandledrtp event".

f. (muxId match and SSRC latch) If packet.muxId is set and receiver =
muxid_table[packet.muxId] is set, route the packet to the receiver and set
ssrc_table[packet.ssrc] = receiver. But if the receiver does not understand
packet.PT, fire the unhandledrtp event.

[BA] If we were allowed to set a pt_table[pt] entry in c), then the last
sentence could be replaced by:
"But if pt_table[packet.PT] is unset or muxid_table[packet.ssrc] !==
pt_table[packet.PT] then fire the
unhandledrtp event".

g. (muxId unmatched) If packet.muxId is set and muxid_table[packet.muxId]
is unset, fire the unhandledrtp event.

[BA] OK.

d. (PT match and SSRC latch) If receiver = pt_table[packet.pt] is set,
route the packet to the receiver and set ssrc_table[packet.ssrc] =
receiver.”

[BA] It is sometimes desirable to allow a sender to switch between
sources, though it only sends one source at a time. In such a scenario we
might route the first source to the receiver, then set
ssrc_table[packet.ssrc] = receiver. When the sender switches sources, will
the new source also be routed to the same receiver via the PT match rule,
with another ssrc_table[packet.ssrc] = receiver entry being created, or
will c) cause the PT match to no longer be operational, resulting in an
unhandledrtp event?

​I think you'd end up with another ssrc_table[packet.ssrc] entry. In other
words, if you have RtpReceiver r1 and pt_table = {101: r1} and yo
u
​receive​
packet p1 = {pt: 101, ssrc: 1001} and p2​ = {p1: 101, ssrc: 2002}, then
you'd end up with a
​ssrc_table = {1001: r1, 2002: r1}, and both packets would get routed to r1.

—
Reply to this email directly or view it on GitHub
#48 (comment).

[pthatcherg]

That sounds pretty good to me. The only thing I'm not sure about is
whether it's worth it to even mention the "receiver_table". Why not just
say "if packet.pt is in receiver.parameters.codec[n].payloadtype"? Why
bother with another table?

On Mon, Jan 12, 2015 at 12:12 PM, aboba notifications@github.com wrote:

Here a draft of proposed text for Section 8.3:

8.3 Matching rules

When the RTCRtpListener object receives an RTP packet over an
RTCDtlsTransport, the RTCRtpListener attempts to determine which
RTCRtpReceiver object to deliver the packet to, based on the values of the
SSRC and payload type fields in the RTP header, as well as the value of the
MID RTP header extension, if present.

The RTCRtpListener maintains four tables in order to facilitate matching:
the ssrc_table which maps SSRC values to RTCRtpReceiver objects; the
muxId_table which maps values of the MID header extension to
RTCRtpReceiverobjects, the pt_table which maps payload type values to
RTCRtpReceiver objects, and the receiver_table which keeps track of payload
types supported by each receiver.

For an RTCRtpReceiver object receiver, table entries are added when
receiver.receive(parameters) is called, and are removed when
receiver.stop() is called. If receiver.receive(parameters) is called again,
all entries referencing receiver are removed prior to adding new entries.

SSRC table: ssrc_table[parameters.encodings[i].ssrc] is set to receiver
for each entry where parameters.encodings[i].ssrc is set, for values of i
from 0 to the number of encodings. If ssrc_table[ssrc] is already set to a
value other than receiver, then receiver.receive(parameters) will throw an
InvalidParameters exception.

muxId table: If parameters.muxId is set, muxId_table[parameters.muxId] is
set to receiver. If muxId_table[muxId] is already set to a value other than
receiver, then receiver.receive(parameters) will throw an InvalidParameters
exception.

payload type table: If parameters.muxId is unset and
parameters.encodings[i].ssrc is unset for all values of i from 0 to the
number of encodings, then add entries to pt_table by setting
pt_table[parameters.codecs[j].payloadType] to receiver, for values of j
from 0 to the number of codecs. If pt_table[pt] is already set to a value
other than receiver, or parameters.codecs[j].payloadType is unset for any
value of j from 0 to the number of codecs, then
receiver.receive(parameters) will throw an InvalidParameters exception.

Receiver table: receiver_table[receiver] is set to an array of payload
types, parameters.codecs[j].payloadtype, where j varies from 0 to the
number of codecs.

When an RTP packet arrives, if ssrc_table[packet.ssrc] is set: set
packet_receiver to ssrc_table[packet.ssrc] and check whether the value of
packet.pt is included in any of the entries in
receiver_table[packet_receiver]. If so, route the packet to
packet_receiver. If packet.pt does not match, unhandledrtp event.

Else if packet.muxId is set: If muxId_table[packet.muxId] is unset, fire
the unhandledrtp event, else set packet_receiver to
muxId_table[packet.muxId] and check whether the value of packet.pt is
included in any of the entries in receiver_table[packet_receiver]. If so,
set ssrc_table[packet.ssrc] to packet_receiver and route the packet to
packet_receiver. If packet.pt does not match, fire the unhandledrtp event.

Else if pt_table[packet.pt] is set: set packet_receiver to pt_table[
packet.pt], set ssrc_table[packet.ssrc] to packet_receiver and route the
packet to packet_receiver. Question: Do we also remove the pt_table[
packet.pt] entry?

Else, fire the unhandledrtp event.

TODO: Revise this paragraph based on the outcome of the discussion on
FEC/RTX/RED.

—
Reply to this email directly or view it on GitHub
#48 (comment).

[pthatcherg]

By the way, it's worth pointing out that at the end of the day, we might
need to simply do what WebRTC 1.0 defines as the rules for using
ssrc/muxId/pt.

On Mon, Jan 26, 2015 at 1:28 PM, Peter Thatcher pthatcher@google.com
wrote:

That sounds pretty good to me. The only thing I'm not sure about is
whether it's worth it to even mention the "receiver_table". Why not just
say "if packet.pt is in receiver.parameters.codec[n].payloadtype"? Why
bother with another table?

On Mon, Jan 12, 2015 at 12:12 PM, aboba notifications@github.com wrote:

Here a draft of proposed text for Section 8.3:

8.3 Matching rules

When the RTCRtpListener object receives an RTP packet over an
RTCDtlsTransport, the RTCRtpListener attempts to determine which
RTCRtpReceiver object to deliver the packet to, based on the values of the
SSRC and payload type fields in the RTP header, as well as the value of the
MID RTP header extension, if present.

The RTCRtpListener maintains four tables in order to facilitate matching:
the ssrc_table which maps SSRC values to RTCRtpReceiver objects; the
muxId_table which maps values of the MID header extension to
RTCRtpReceiverobjects, the pt_table which maps payload type values to
RTCRtpReceiver objects, and the receiver_table which keeps track of payload
types supported by each receiver.

For an RTCRtpReceiver object receiver, table entries are added when
receiver.receive(parameters) is called, and are removed when
receiver.stop() is called. If receiver.receive(parameters) is called again,
all entries referencing receiver are removed prior to adding new entries.

SSRC table: ssrc_table[parameters.encodings[i].ssrc] is set to receiver
for each entry where parameters.encodings[i].ssrc is set, for values of i
from 0 to the number of encodings. If ssrc_table[ssrc] is already set to a
value other than receiver, then receiver.receive(parameters) will throw an
InvalidParameters exception.

muxId table: If parameters.muxId is set, muxId_table[parameters.muxId] is
set to receiver. If muxId_table[muxId] is already set to a value other than
receiver, then receiver.receive(parameters) will throw an InvalidParameters
exception.

payload type table: If parameters.muxId is unset and
parameters.encodings[i].ssrc is unset for all values of i from 0 to the
number of encodings, then add entries to pt_table by setting
pt_table[parameters.codecs[j].payloadType] to receiver, for values of j
from 0 to the number of codecs. If pt_table[pt] is already set to a value
other than receiver, or parameters.codecs[j].payloadType is unset for any
value of j from 0 to the number of codecs, then
receiver.receive(parameters) will throw an InvalidParameters exception.

Receiver table: receiver_table[receiver] is set to an array of payload
types, parameters.codecs[j].payloadtype, where j varies from 0 to the
number of codecs.

When an RTP packet arrives, if ssrc_table[packet.ssrc] is set: set
packet_receiver to ssrc_table[packet.ssrc] and check whether the value of
packet.pt is included in any of the entries in
receiver_table[packet_receiver]. If so, route the packet to
packet_receiver. If packet.pt does not match, unhandledrtp event.

Else if packet.muxId is set: If muxId_table[packet.muxId] is unset, fire
the unhandledrtp event, else set packet_receiver to
muxId_table[packet.muxId] and check whether the value of packet.pt is
included in any of the entries in receiver_table[packet_receiver]. If so,
set ssrc_table[packet.ssrc] to packet_receiver and route the packet to
packet_receiver. If packet.pt does not match, fire the unhandledrtp
event.

Else if pt_table[packet.pt] is set: set packet_receiver to pt_table[
packet.pt], set ssrc_table[packet.ssrc] to packet_receiver and route the
packet to packet_receiver. Question: Do we also remove the pt_table[
packet.pt] entry?

Else, fire the unhandledrtp event.

TODO: Revise this paragraph based on the outcome of the discussion on
FEC/RTX/RED.

—
Reply to this email directly or view it on GitHub
#48 (comment).

[aboba]

Agree that ORTC API needs to behave the same way as WebRTC 1.0. The
question is where this is defined, exactly. I remember us talking about it
in IETF RTCWEB, but can't seem to find the document in which the
conclusions appeared (just checked the JSEP draft).

On Tue, Jan 27, 2015 at 9:01 AM, pthatcherg notifications@github.com
wrote:

By the way, it's worth pointing out that at the end of the day, we might
need to simply do what WebRTC 1.0 defines as the rules for using
ssrc/muxId/pt.

On Mon, Jan 26, 2015 at 1:28 PM, Peter Thatcher pthatcher@google.com
wrote:

That sounds pretty good to me. The only thing I'm not sure about is
whether it's worth it to even mention the "receiver_table". Why not just
say "if packet.pt is in receiver.parameters.codec[n].payloadtype"? Why
bother with another table?

On Mon, Jan 12, 2015 at 12:12 PM, aboba notifications@github.com
wrote:

Here a draft of proposed text for Section 8.3:

8.3 Matching rules

When the RTCRtpListener object receives an RTP packet over an
RTCDtlsTransport, the RTCRtpListener attempts to determine which
RTCRtpReceiver object to deliver the packet to, based on the values of
the
SSRC and payload type fields in the RTP header, as well as the value of
the
MID RTP header extension, if present.

The RTCRtpListener maintains four tables in order to facilitate
matching:
the ssrc_table which maps SSRC values to RTCRtpReceiver objects; the
muxId_table which maps values of the MID header extension to
RTCRtpReceiverobjects, the pt_table which maps payload type values to
RTCRtpReceiver objects, and the receiver_table which keeps track of
payload
types supported by each receiver.

For an RTCRtpReceiver object receiver, table entries are added when
receiver.receive(parameters) is called, and are removed when
receiver.stop() is called. If receiver.receive(parameters) is called
again,
all entries referencing receiver are removed prior to adding new
entries.

SSRC table: ssrc_table[parameters.encodings[i].ssrc] is set to receiver
for each entry where parameters.encodings[i].ssrc is set, for values of
i
from 0 to the number of encodings. If ssrc_table[ssrc] is already set
to a
value other than receiver, then receiver.receive(parameters) will throw
an
InvalidParameters exception.

muxId table: If parameters.muxId is set, muxId_table[parameters.muxId]
is
set to receiver. If muxId_table[muxId] is already set to a value other
than
receiver, then receiver.receive(parameters) will throw an
InvalidParameters
exception.

payload type table: If parameters.muxId is unset and
parameters.encodings[i].ssrc is unset for all values of i from 0 to the
number of encodings, then add entries to pt_table by setting
pt_table[parameters.codecs[j].payloadType] to receiver, for values of j
from 0 to the number of codecs. If pt_table[pt] is already set to a
value
other than receiver, or parameters.codecs[j].payloadType is unset for
any
value of j from 0 to the number of codecs, then
receiver.receive(parameters) will throw an InvalidParameters exception.

Receiver table: receiver_table[receiver] is set to an array of payload
types, parameters.codecs[j].payloadtype, where j varies from 0 to the
number of codecs.

When an RTP packet arrives, if ssrc_table[packet.ssrc] is set: set
packet_receiver to ssrc_table[packet.ssrc] and check whether the value
of
packet.pt is included in any of the entries in
receiver_table[packet_receiver]. If so, route the packet to
packet_receiver. If packet.pt does not match, unhandledrtp event.

Else if packet.muxId is set: If muxId_table[packet.muxId] is unset,
fire
the unhandledrtp event, else set packet_receiver to
muxId_table[packet.muxId] and check whether the value of packet.pt is
included in any of the entries in receiver_table[packet_receiver]. If
so,
set ssrc_table[packet.ssrc] to packet_receiver and route the packet to
packet_receiver. If packet.pt does not match, fire the unhandledrtp
event.

Else if pt_table[packet.pt] is set: set packet_receiver to pt_table[
packet.pt], set ssrc_table[packet.ssrc] to packet_receiver and route
the
packet to packet_receiver. Question: Do we also remove the pt_table[
packet.pt] entry?

Else, fire the unhandledrtp event.

TODO: Revise this paragraph based on the outcome of the discussion on
FEC/RTX/RED.

—
Reply to this email directly or view it on GitHub
#48 (comment).

—
Reply to this email directly or view it on GitHub
#48 (comment).

[pthatcherg]

https://tools.ietf.org/html/draft-roach-mmusic-unified-plan-00#section-3.2

On Wed, Jan 28, 2015 at 8:04 AM, aboba notifications@github.com wrote:

Agree that ORTC API needs to behave the same way as WebRTC 1.0. The
question is where this is defined, exactly. I remember us talking about it
in IETF RTCWEB, but can't seem to find the document in which the
conclusions appeared (just checked the JSEP draft).

On Tue, Jan 27, 2015 at 9:01 AM, pthatcherg notifications@github.com
wrote:

By the way, it's worth pointing out that at the end of the day, we might
need to simply do what WebRTC 1.0 defines as the rules for using
ssrc/muxId/pt.

On Mon, Jan 26, 2015 at 1:28 PM, Peter Thatcher pthatcher@google.com
wrote:

That sounds pretty good to me. The only thing I'm not sure about is
whether it's worth it to even mention the "receiver_table". Why not
just
say "if packet.pt is in receiver.parameters.codec[n].payloadtype"?
Why
bother with another table?

On Mon, Jan 12, 2015 at 12:12 PM, aboba notifications@github.com
wrote:

Here a draft of proposed text for Section 8.3:

8.3 Matching rules

When the RTCRtpListener object receives an RTP packet over an
RTCDtlsTransport, the RTCRtpListener attempts to determine which
RTCRtpReceiver object to deliver the packet to, based on the values
of
the
SSRC and payload type fields in the RTP header, as well as the value
of
the
MID RTP header extension, if present.

The RTCRtpListener maintains four tables in order to facilitate
matching:
the ssrc_table which maps SSRC values to RTCRtpReceiver objects; the
muxId_table which maps values of the MID header extension to
RTCRtpReceiverobjects, the pt_table which maps payload type values to
RTCRtpReceiver objects, and the receiver_table which keeps track of
payload
types supported by each receiver.

For an RTCRtpReceiver object receiver, table entries are added when
receiver.receive(parameters) is called, and are removed when
receiver.stop() is called. If receiver.receive(parameters) is called
again,
all entries referencing receiver are removed prior to adding new
entries.

SSRC table: ssrc_table[parameters.encodings[i].ssrc] is set to
receiver
for each entry where parameters.encodings[i].ssrc is set, for values
of
i
from 0 to the number of encodings. If ssrc_table[ssrc] is already set
to a
value other than receiver, then receiver.receive(parameters) will
throw
an
InvalidParameters exception.

muxId table: If parameters.muxId is set,
muxId_table[parameters.muxId]
is
set to receiver. If muxId_table[muxId] is already set to a value
other
than
receiver, then receiver.receive(parameters) will throw an
InvalidParameters
exception.

payload type table: If parameters.muxId is unset and
parameters.encodings[i].ssrc is unset for all values of i from 0 to
the
number of encodings, then add entries to pt_table by setting
pt_table[parameters.codecs[j].payloadType] to receiver, for values of
j
from 0 to the number of codecs. If pt_table[pt] is already set to a
value
other than receiver, or parameters.codecs[j].payloadType is unset for
any
value of j from 0 to the number of codecs, then
receiver.receive(parameters) will throw an InvalidParameters
exception.

Receiver table: receiver_table[receiver] is set to an array of
payload
types, parameters.codecs[j].payloadtype, where j varies from 0 to the
number of codecs.

When an RTP packet arrives, if ssrc_table[packet.ssrc] is set: set
packet_receiver to ssrc_table[packet.ssrc] and check whether the
value
of
packet.pt is included in any of the entries in
receiver_table[packet_receiver]. If so, route the packet to
packet_receiver. If packet.pt does not match, unhandledrtp event.

Else if packet.muxId is set: If muxId_table[packet.muxId] is unset,
fire
the unhandledrtp event, else set packet_receiver to
muxId_table[packet.muxId] and check whether the value of packet.pt
is
included in any of the entries in receiver_table[packet_receiver]. If
so,
set ssrc_table[packet.ssrc] to packet_receiver and route the packet
to
packet_receiver. If packet.pt does not match, fire the unhandledrtp
event.

Else if pt_table[packet.pt] is set: set packet_receiver to pt_table[
packet.pt], set ssrc_table[packet.ssrc] to packet_receiver and route
the
packet to packet_receiver. Question: Do we also remove the pt_table[
packet.pt] entry?

Else, fire the unhandledrtp event.

TODO: Revise this paragraph based on the outcome of the discussion on
FEC/RTX/RED.

—
Reply to this email directly or view it on GitHub
#48 (comment).

—
Reply to this email directly or view it on GitHub
#48 (comment).

—
Reply to this email directly or view it on GitHub
#48 (comment).

[robin-raymond]

https://datatracker.ietf.org/doc/draft-ietf-mmusic-sdp-bundle-negotiation/?include_text=1

Section 9+10 are applicable.

[aboba]

Looking at draft-roach-mmusic-unified-plan Section 3.2, it would appear that the muxId and ssrc tables from Peter's proposal are consistent with the advise given there. The concerns so far seem to arise with respect to payload type demultiplexing, covered in Section 3.2.1.2. Since Peter's proposal requires PT uniqueness, I believe it behaves as though the constraint is set in Section 3.2.1.2 below:

3.2.1.2. Payload Type Correlation

To support implementations that cannot implement the RTP header
extension described in Section 3.2.1.1 but which wish to use the
BUNDLE mechanism, we allow an alternate (but less-preferred) means of
correlation using payload type. This approach takes advantage of the
fact that the offer contains payload types chosen by its creator,
which will be present in any RTP received from the remote party. If
these payload types are unique, then they can be used to reliably
correlate incoming RTP streams to their m= lines.

Because of its inherent limitations, it is advisable to use other
correlation techniques than PT multiplexing if at all possible. In
order to accomplish this, we propose, for WebRTC, that use of this
technique be controlled by an additional constraint passed to
createOffer by the Web application.

If this constraint is set, the browser MUST behave as described in
this section. If the constraint is not set, the browser MUST use
identical PTs for the same codec values within each m-line bundle.

When such a constraint is present, implementations attempt to
entirely exhaust the dynamic payload type numbering space before re-
using a payload type within the scope of a local transport address.
If such a constraint is present and the payload type space would
ordinarily be exhausted within the scope of a local transport
address, the implementation MAY (at its discretion) take any of the
following actions:

1. Bind to multiple local transport addresses (using different
   BUNDLE groups) for the purpose of keeping the {payload type,
   transport address} combination unique.
2. Signal a failure to the application.

[pthatcherg]

I think my proposal is very similar to this draft in the sense that if you
set either the muxID or the SSRC, it's implied that the PTs don't matter
for demux, just like if this constraint mentioned here were off. If muxID
and SSRC are not set, then it's as though the constraint is set (PTs must
be non-overlapping between RtpSenders). So, the behavior between the two
is quite similar.

The part that seems difficult for us is what we do about SSRC latching, and
the big question is: if you are doing PT demux and you get two packets
with the same PT and different SSRCs, what do you do? Do you route them
both to the same RtpSender, or do you drop the second one? Or something
else? This draft seems to imply that we do route it to the same
RtpSender. If so, I'd suggest we just remove all mentions of "ssrc
latching" from my proposal and match what this draft is doing. On the
other hand, I might be missing something in the draft about this.

On Thu, Jan 29, 2015 at 7:01 AM, aboba notifications@github.com wrote:

Looking at draft-roach-mmusic-unified-plan Section 3.2, it would appear
that the muxId and ssrc tables from Peter's proposal are consistent with
the advise given there. The concerns so far seem to arise with respect to
payload type demultiplexing, covered in Section 3.2.1.2. Since Peter's
proposal requires PT uniqueness, I believe it behaves as though the
constraint is set in Section 3.2.1.2 below:

3.2.1.2. Payload Type Correlation

To support implementations that cannot implement the RTP header
extension described in Section 3.2.1.1 but which wish to use the
BUNDLE mechanism, we allow an alternate (but less-preferred) means of
correlation using payload type. This approach takes advantage of the
fact that the offer contains payload types chosen by its creator,
which will be present in any RTP received from the remote party. If
these payload types are unique, then they can be used to reliably
correlate incoming RTP streams to their m= lines.

Because of its inherent limitations, it is advisable to use other
correlation techniques than PT multiplexing if at all possible. In
order to accomplish this, we propose, for WebRTC, that use of this
technique be controlled by an additional constraint passed to
createOffer by the Web application.

If this constraint is set, the browser MUST behave as described in
this section. If the constraint is not set, the browser MUST use
identical PTs for the same codec values within each m-line bundle.

When such a constraint is present, implementations attempt to
entirely exhaust the dynamic payload type numbering space before re-
using a payload type within the scope of a local transport address.
If such a constraint is present and the payload type space would
ordinarily be exhausted within the scope of a local transport
address, the implementation MAY (at its discretion) take any of the
following actions:

Bind to multiple local transport addresses (using different
BUNDLE groups) for the purpose of keeping the {payload type,
transport address} combination unique.
2.

Signal a failure to the application.

—
Reply to this email directly or view it on GitHub
#48 (comment).

[aboba]

Here is an analysis from Peter Thatcher of several options for handling two use cases:

• Use case 1 is a simple O/A involving a single payload type as well as RTX.
• Use case 2 is a conference bridge with N participants that forwards (not mixes) audio packets, all on the same payload type.

Option A: Only 1 SSRC per RtpReceiver gets routed because of PT, after that it's unhandled
Option B: All SSRCs get routed because of PT, nothing unhandled
Option C: Only 1 SSRC per RtpReceiver, plus 1 for RTX per RtpReceiver
Option D: Have a .routeAllSsrcs/routeOnlyOneSsrc.
Option E: B + and a .handledNewSsrc event on RtpReceiver
Option F: A + and a .receiving/not receiving event on RtpReceiver
Option G: B + add .onlyOneSsrc or .handledNewSsrc or something when we find that the use case 2 is a real world problem.
Option H: RtpReceiver w/ 0 encodings == route all. RtpReceiver with 1 encoding w/o SSRC == route one.
receiver.setParameters({codecs: [...]}) // receive all SSRCs
receiver.setParameters({codecs: [...], encodings: []) // receive nothing :)
receiver.setParameters({cocecs: [...], encodings: [{active:false}]) // latch one primary SSRC, one RTX SSRC, one FEC SSRC

"muxID or PTs choose RtpReceiver"
"encodings without SSRC latch to one primary SSRC (and one RTX ssrc); undefined encodings list means auto-add encodings (matching the PT/muxID)"

Use case 1: Simple case with RTX.
One RtpReceiver, two unsignalled SSRCs: one for primary, and one for RTX.

Option A: Have to listen to unhandled, but then when you add that one, the first one gets lost..
Option B: Works.
Option C: Works.
Option D: Works with .routeAllSsrcs.
Option E: Works.
Option H: Works with no encodings specified

Use case 2: Conference audio.
N RtpReceivers, all unsignalled SSRCS (one each), all same PTs

Option A: Works.
Option B: Have to have the first RtpReceiver be built with SSRC using unhandled. May have clippling. Or signal SSRCs or muxID.
Option C: Works.
Option D: Works with .routeOnlyOneSsrc.
Option E: Make a default RtpReciever, and add new ones as they come in. Much less clipping.
Option H: Works with 1 encoding specified with no SSRC

[aboba]

Recent comment from Magnus relating to JSEP that may be relevant to this issue:
http://www.ietf.org/mail-archive/web/rtcweb/current/msg14700.html

[aboba]

I would like to propose that we close this general bug, and work on the more specific ones (e.g. relating to RED/RTX/FEC).