RTX APT capability?

[robin-raymond]

Since we don't have a capability for RTX APT, a single RTX codec gets listed allocating a single preferredPayloadType. The trouble is that payload type is not relevant because an RTX codec is needed per media codec that supports RTX. Likewise, there's no guarantee the engine supports RTX for each media codec.

Currently a developer would need to disregard the RTX codec, and allocate their own payload per RTX codec per media codec they wish to use with RTX. This is certainly doable but problematic. There is currently no way for the developer to know which media codecs support RTX or not.

I suspect this issue may exist for RED + ULPFEC, FlexFec too possibly. Although there's an advantage that each of those only need to allocate a single codec per clock rate (as the payload contains the original media codec) whereas that's not true of RTX payloads.

[ibc]

getCapabilities() may retrieve as many RTX named codecs as media types supporting RTX. Each of these RTX codec should include a options.apt or similar field pointing to the preferredPayloadType of the corresponding media codec.

But again: These things happen because we are considering RTX a codec rather than a "real-codec feature".

[robin-raymond]

@ibc we could reverse this and make each codec announce it's corresponding RTX payload type instead of having apt inside the RTX codec and announcing RTX as a codec

So in CodecCapability / CodecParameters would have an rtxPayloadType value (optionally) and remove the RTX codec entirely.

[ibc]

That's exactly what I meant with "real-codec feature" :)

And we can do the same with FEC.

[murillo128]

Not sure if I understand the problem. RTX is codec-unaware, so either it is supported on RTPSender/RTPReceive or not. So in terms on capabilities, if RTX is supported, it is supported for all codecs.

Another thing is if you want to use it for some specific codecs or not (associating an ssrc).

[ibc]

So in terms on capabilities, if RTX is supported, it is supported for all codecs.

Yep. The point is that, in case the browser supports RTX, every real codec retrieved via getCapabilities() would include a rtxPayloadType numeric property.

[murillo128]

Yep. The point is that, in case the browser supports RTX, every real codec retrieved via getCapabilities() would include a rtxPayloadType numeric property.

Indeed

[robin-raymond]

In theory RTX is universal for all codecs, but in practice, it's not always offered for all codecs. So there needs to be a way to make it optional property if we do it this way.

[ibc]

there needs to be a way to make it optional property

Just don't set the corresponding rtpCodecParameters.rtxPayloadType property when calling rtpSender.send().

[robin-raymond]

@ibc that's all true, but I'm being specific about that because if we are going to issue any "fixes" we need to be 100% clear about everything to the dotted i and crossed t.

[robin-raymond]

@pthatcher do you have an opinion on this issue?

I think this is applicable to FEC too.

[aboba]

@pthatcher @robin-raymond

Related WebRTC 1.0 Issue: w3c/webrtc-pc#548

[robin-raymond]

Just to be clear, I'm in favour of this change to make RTX and FEC as options under the "main" real codecs but I think we need to get buy in for such a change. I find implicit cross reference meanings between codecs and their RTX/FEC counterparts in the codec list is very problematic and cumbersome (.e.g. making sure clock rate matches, settings match and ambiguity where multiple matches are possible, RED listed with or without payload types, multiple REDs with ambiguous choices, etc). I further think this is applicable to both RTX and FEC.

Plus, it's not just an RTX preferred payload type. RTX also has rtxTime. So if we put this under the CodecCapability / or CodecParameter we'd have to include these values as appropriate.

For example:

partial dictionary RTCRtpCodecCapability {
   RTCRtxCodecCapability           rtx;
   sequence<RTCFecCodecCapability> fec;
};

dictionary RTCRtpCodecCapability {
   payloadtype      preferredPayloadType;
   unsigned long    rtxTime;    // recommended rtx time value
}

dictionary RTCFecCodecCapability {
   DOMString        mechanism;
   payloadtype      preferredPayloadType; // if applicable, e.g. RED+ULPFEC
   Dictionary       parameters;    // optional fec specific capabilities (which do exist for some codecs)
}

partial dictionary RTCRtpCodecParameters {
   RTCRtxCodecParameters           rtx;
   sequence<RTCFecCodecParameters> fec;  // maybe need a sequence to support multiple mechanisms simultaneously
};

dictionary RTCRtxCodecParameters {
   payloadtype      payloadType;
   unsigned long    rtxTime;
}

dictionary RTCFecCodecParameters {
   DOMString        mechanism;
   payloadtype      payloadType;   // if applicable, e.g. RED + ULPFEC
   Dictionary       parameters;    // optional fec specific parameters
}

[ibc]

@robin-raymond IMHO that's exactly the way to go. Then we can also refactor current encodings which IMHO should be simplified A LOT. Something like:

dictionary RTCRtpParameters {
    DOMString                                 muxId = "";
    sequence<RTCRtpCodecParameters>           codecs;
    sequence<RTCRtpHeaderExtensionParameters> headerExtensions;
    RTCRtpEncodingParameters                  encodings;  // Not a sequence anymore
    RTCRtcpParameters                         rtcp;
    RTCDegradationPreference                  degradationPreference = "balanced";
};

partial dictionary RTCRtpEncodingParameters {
    unsigned long       mediaSsrc;
    unsigned long       rtxSsrc;
    unsigned long       fecSsrc;
    [...]
};

[murillo128]

You need too support simulcast.. :)

[murillo128]

what I don't understand is why don't we set the codec inside the encoding:

dictionary RTCRtpParameters {
    DOMString                                 muxId = "";
    sequence<RTCRtpHeaderExtensionParameters> headerExtensions;
    sequence<RRTCRtpEncodingParameters>                 encodings; 
    RTCRtcpParameters                         rtcp;
    RTCDegradationPreference                  degradationPreference = "balanced";
};

partial dictionary RTCRtpEncodingParameters {
    unsigned long       mediaSsrc;
    unsigned long       rtxSsrc;
    unsigned long       fecSsrc;
    sequence<RTCRtpCodecParameters>           codecs;
    [...]
};

Then we don't need to reference the payolad type anymore as it would be inside the codec parameters, right?