[RtpParameters] clockRate is per stream, not per codec #404

Closed
ibc opened this Issue Mar 4, 2016 · 16 comments

Projects

None yet

3 participants

@ibc
Contributor
ibc commented Mar 4, 2016

What is the purpose of per-codec clockRate when that is a parameter related to the whole stream? What would happen if two codecs in the same RtpSender have different clockRate?

IMHO RtpParameters maps "too much" the bad way in which SDP signals the stuff.

http://ortc.org/wp-content/uploads/2015/11/ortc.html#idl-def-RTCRtpCodecParameters

/cc @murillo128

@robin-raymond
Contributor

Within the same RtpSender is fine since they will have different payloadType values if you have two encodings pointing to two codecs with two different clock rates (and thus are simulcast with different clock rates). As for too SDP like, this was to help aid the translation to/from adapter shims. Not saying it's ideal but that's why...

@aboba
Contributor
aboba commented Mar 9, 2016

@robin-raymond You cannot have two distinct clock rates sharing an SSRC according to RFC 7160:
https://datatracker.ietf.org/doc/rfc7160/

So if encodings[a].ssrc === encodings[b].ssrc then encodings[a].codecPayloadType better have the same clockrate as encodings[b].codecPayloadType.

@aboba
Contributor
aboba commented Mar 9, 2016

@ibc I think that two codecs in the same RtpSender can have a different clockrate. As long as the values of encodings[].ssrc are different. Now in practice this implies that the SSRC will change if the codecs are reordered, so the matching rules and RtpReceiver would need to deal with that.

@ibc
Contributor
ibc commented Mar 9, 2016

two codecs in the same RtpSender can have a different clockrate. As long as the values of encodings[].ssrc are different

That's nice. However it opens again the door to inconsistent RtpParameters. What would happen if two codecs with different clock rate share the same SSRC in their corresponding encoding?

@robin-raymond
Contributor

@aboba and you won't share the same SSRC if its simulcasting. If you don't specify and SSRC the system will choose one. @ibc You can't share an SSRC across encodings.

@ibc
Contributor
ibc commented Mar 9, 2016

You can't share an SSRC across encodings.

Really? Then how is ORTC supposed to map this?:

a=rtpmap:100 VP8/90000
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=rtcp-fb:100 goog-remb
a=rtcp-fb:100 transport-cc
a=rtpmap:101 VP9/90000
a=rtcp-fb:101 ccm fir
a=rtcp-fb:101 nack
a=rtcp-fb:101 nack pli
a=rtcp-fb:101 goog-remb
a=rtcp-fb:101 transport-cc
a=rtpmap:116 red/90000
a=rtpmap:117 ulpfec/90000
a=rtpmap:96 rtx/90000
a=fmtp:96 apt=100
a=rtpmap:97 rtx/90000
a=fmtp:97 apt=101
a=rtpmap:98 rtx/90000
a=fmtp:98 apt=116
a=ssrc-group:FID 2224031971 3254585230
a=ssrc:2224031971 cname:oC/i06PA+Lda+t1P
a=ssrc:3254585230 cname:oC/i06PA+Lda+t1P

Both VP8 (PT 100) and VP9 (PT 101) share the same SSRC (let's assume it is 2224031971) and RTX for the 2224031971 stream is carried over another stream with SSRC 3254585230.

And it happens that there is RTX for both VP8 and VP9 (along with other codecs), so I need two encodings:

  • For VP8:
codecPayloadType: 100
ssrc: 2224031971
rtx:
    payloadType: 96
    ssrc: 3254585230
  • For VP9:
codecPayloadType: 101
ssrc: 2224031971
rtx:
    payloadType: 97
    ssrc: 3254585230

So two encodings with same SSRC.

@ibc
Contributor
ibc commented Mar 9, 2016

Another solution would be that RtpEncodingParameteters has a rtxs field which is a sequence of RtpRtxParameters rather than just one.

@aboba
Contributor
aboba commented Mar 9, 2016

@ibc @robin-raymond Having VP8, VP9, H.264, etc. all using the same SSRC is not illegal because they all share the same clock rate (90000). The SDP that Inaki included is generated by createOffer() in recent Chrome Canary builds, so if we want the shim to work for video it needs to be supported.

@aboba
Contributor
aboba commented Mar 9, 2016

@ibc If two codecs with different clock rate share the same SSRC in their corresponding encoding, then the problems described in RFC 7160 will arise. Currently, the spec does not indicate that this should cause an exception, but perhaps we should say that.

@ibc
Contributor
ibc commented Mar 9, 2016

The SDP that Inaki included is generated by createOffer() in recent Chrome Canary builds, so if we want the shim to work for video it needs to be supported.

I don't think we need such a new SDP to justify this usecase. Even if I just send ULAW and ALAW I need to create two encodings to set the proper SSRC (which is the same) for both codecs. So the only I say is that:

  • encoding.codecPayloadType must be set, unique and must match a payloadType in codecs.
  • encoding.ssrc can be the same for different encodings.
@robin-raymond
Contributor

@ibc I think the matching rules do not handle this right now. That's why I said this... I think they need to be fixed.

My general matching works by requiring everything that is "set" must match. If it is "unset" then it can be figured out and matches according to order added. So if you specify SSRC and PT then both must match. If only one entry set then that must match. Once a match is had then the "unset" property is filled in.

Thus in your use case you can have one encoding with the SSRC set with a PT unset. Then it will match either VP8 or VP9. Or you could have two encodings with SSRC and PT set and thus only one will match based on the combination.

Once I find the best match then I add a routing entry for an SSRC so it fast routes to the receiver based on SSRC alone, as I describe in #368

I think this is more flexible this way and handles your concerns quite well.

@ibc
Contributor
ibc commented Mar 10, 2016

Thanks. I agree and all is fine now. Of course I would like that ORTC spec is fixed regarding this subject. I can do that and send a PR this weekend.

@aboba aboba added question 1.1 and removed question labels Mar 12, 2016
@aboba
Contributor
aboba commented Mar 12, 2016

This is related to the following Issues:
#408
#386
#368
#362

@ibc
Contributor
ibc commented Mar 13, 2016

RFC 7160:

3.2. Same SSRC

The simplest way to manage multiple clock rates is to use the same
SSRC for all of the payload types regardless of the clock rates.
Unfortunately, there is no clear definition on how the RTP timestamp
should be calculated in this case.

Is this a real issue?

a=rtpmap:111 opus/48000/2
a=fmtp:111 minptime=10; useinbandfec=1
a=rtpmap:103 ISAC/16000
a=rtpmap:104 ISAC/32000
a=rtpmap:9 G722/8000
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:106 CN/32000
a=rtpmap:105 CN/16000
a=rtpmap:13 CN/8000

So, if in the middle of a session the browser changes from sending opus/48000 to PCMA/8000, would
the RTP timestamp calculation become crazy?

BTW:

The "clock rate" is the rate at which the timestamp in the RTP header is incremented, which need not be the same as the codec's sampling rate. For instance, video codecs typically use a clock rate of 90000 so their frames can be more precisely aligned with the RTCP NTP timestamp, even though video sampling rates are typically in the range of 1–60 samples per second.

I think ORTC should not impose constraints to the cockRate values within a RtpCodecsParameters object. Let implementations deal with RFC 7160 if they wish to.

@aboba aboba added the PR exists label Mar 15, 2016
@aboba
Contributor
aboba commented Mar 15, 2016

@ibc @robin-raymond

I have submitted an RTX/RED/FEC example in PR #422

This example is definitely not straight forward, particularly with respect to how to indicate retransmission of RED/FEC.

We should probably discuss this in an ORTC CG meeting.

@aboba aboba closed this Mar 16, 2016
@ibc
Contributor
ibc commented Mar 17, 2016

@aboba thanks for the example. It looks good for me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment