If a preferred codec is filtered out, does it still get assigned a PT?

[henbos]

Use case: I want to signal in the SDP that I support sending X, but my transceiver is sendrecv.

X is filtered out from the preference list at createOffer, so it's not in the m=video line. But should it be in the a=rtpmap list?

Arguments for:

• It's useful and avoids having to signal things outside SDP.

Argument against:

• JSEP: "media formats MUST be generated in the corresponding order and MUST exclude any codecs not present in the codec preferences"

[alvestrand]

I think JSEP did not think about this use case.
For subsequent offers, the rule is different:

The "m=" line and corresponding "a=rtpmap" and "a=fmtp" lines MUST only include media formats that have not been excluded by the codec preferences of the associated transceiver and also MUST include all currently available formats. Media formats that were previously offered but are no longer available (e.g., a shared hardware codec) MAY be excluded.

I think JSEP should have an errata suggesting a replacement with "MUST be generated in the corresponding order and MUST include all codecs present in the codec preferences, plus any codecs that need a PT assigned (for example a send only codec)".

[henbos]

That errata makes sense to me.

Though we need to have setCodecPreferences not throw for send only codecs for this to work. I thought SCP only rejected if there was no recv codec in the list, but it currently throws if any codec is sendonly even if you have receive codecs in the list too.

Let codecCapabilities be RTCRtpReceiver.getCapabilities(kind).codecs.

For each codec in codecs,

If codec does not match any codec in codecCapabilities, throw InvalidModificationError.

[fippo]

I believe JSEP had a specific intention (but we should really nag the authors).
If your initial offer included a codec that you were capable of sending but not receiving how would you signal that?

The "m=" line and corresponding "a=rtpmap" and "a=fmtp" lines MUST only include ..

That covers the following scenario:
A: H264-only offer
B: H264-only answer (but might include additional codecs, unless sCP was used)
B: H264 + VP8 offer

Do we have tests for that? I doubt it but that is fixable.

[rshpount]

I think the general intent with the SDP offer/answer is to include codecs that you can only send in the sendonly m= line, codecs that can only receive on the receiveonly m= line, and only list codecs you can both send and receive on the sendrecv m= line. SDP is not designed to negotiate different codecs being used for sending and receiving on the same m= line.

[henbos]

If we go with #2939 I think this largely becomes a non-issue and we'd be consistent with @rshpount's interpretation of SDP. E.g. if you want to signal it you restrict transceiver direction and if you don't you can always add it in the answer as a new PT but that's less straightforward use of the APIs.

[cdh4u]

I think the general intent with the SDP offer/answer is to include codecs that you can only send in the sendonly m= line, codecs that can only receive on the receiveonly m= line, and only list codecs you can both send and receive on the sendrecv m= line. SDP is not designed to negotiate different codecs being used for sending and receiving on the same m= line.

Correct.

[aboba]

Yes. It is laid out in RFC 3264 Section 5.1:

"For a sendonly stream, the offer SHOULD indicate those formats the offerer is willing to send for this stream. For a recvonly stream, the offer SHOULD indicate those formats the offerer is willing to receive for this stream. For a sendrecv stream, the offer SHOULD indicate those codecs that the offerer is willing to send and receive with."

[alvestrand]

@rshpount we seem to have current applications that have sendonly codecs that they expect to be able to use for sendrecv lines - in webrtc scenarios, the media streams are treated as unidirectional, but can be paired up to use a sendrecv media section, without that creating any other binding between them - EXCEPT that they now share a set of codecs.

If it was feasible to outlaw sendrecv m-lines under Unified Plan, I'd be happy to do so - but I don't think that's Web compatible at this point.

[rshpount]

@alvestrand The right thing would be to get rid of SDP and offer/answer and replace them with an API service that specifies which codecs should be sent and received from each transceiver. This is, obviously, a breaking change.

The issue with the offer/answer is that between the time the offer is generated and the answer is received, the offerer should expect to receive any of the codecs listed in the sendrecv m= line. Once the answer is received, the offerer can discard all the codecs not included in the answer for the sendrecv line, which means using codecs not included in the answer is not allowed in either direction. If the codec is present in the answer sendrecv m= line, the sender can send it. It does not matter that the codec is not the first in the sequence. The sequence of codecs in the m= line is just a preference but not a hard requirement. Also, the answerer can add codecs in the answer that were not present in the offer. The offerer can send media using these codecs as well. Finally, the answerer can send using any of the codecs present in both the offer and answer and switch between them as appropriate. The only way for the offerer to force a smaller set from the original set was to initiate another offer/answer.

So, what this comes down to is that an offer/answer negotiation can set up a list of common codecs (with a small caveat that the answerer can add unidirectional codecs that it can receive). You can add an API surface to force sending using a specific codec from the negotiated list. That API is not a codec preference. It would be a straight codec selection.

[alvestrand]

@rshpount this (getting to an SDP-free future) is part of the motivation behind #2935 - getting to a description of the codec situation that is SDP-independent.

The API surface for setting a specific sending codec is documented in https://w3c.github.io/webrtc-extensions/#dom-rtcrtpencodingparameters-codec - invoked by SetParameters() on the sender, just as one would expect in an SDP-free environment. It does not trigger "negotiation needed".

The question that this issue is raising is the case where, for codec "S", which we only support for sending, how, in an initial offer, do we allocate a PT for that and request that the responder signal its willingness to accept that codec?

This offer will do it (as long as the responder supports S and hasn't set codec preferences):

m=video .... 94 ....
a=sendrecv
a=rtpmap:94:S

But it ALSO tells the receiver that we're willing to receive S, which is not true.

Stating that this is impossible in SDP and we're OK with that is fine with me; there are worse things in the world.
But it should be an explicit decision to say that this is not possible.

[alvestrand]

My current proposal:

Add a NOTE saying:

"NOTE: Using SDP, it is not possible to indicate in a media section with direction "sendrecv" of an offer that you support a codec for sending that you don't support for receiving. This is a limitation of SDP."

[stefhak]

I agree to your conclusion @alvestrand and like the proposed note. Likely there should be some reference to an SDP RFC as well, but that's just a detail.

[rshpount]

I agree with the note. I would expand it a little bit, saying that:
"NOTE: Using SDP, it is not possible to indicate in a media section with the direction "sendrecv" of an offer that you support a codec for sending that you don't support for receiving. This is a limitation of SDP. Such information should be conveyed via a side channel, outside of the SDP offer/answer negotiation."

[dontcallmedom-bot]

This issue was mentioned in WEBRTCWG-2024-02-20 (Page 18)

[alvestrand]

Should we add to the note ", or sendonly / recvonly media sections can be used"?
There's no problem with signalling the unidirectional codecs if the direction is known.

[cdh4u]

I agree with the note. I would expand it a little bit, saying that: "NOTE: Using SDP, it is not possible to indicate in a media section with the direction "sendrecv" of an offer that you support a codec for sending that you don't support for receiving. This is a limitation of SDP. Such information should be conveyed via a side channel, outside of the SDP offer/answer negotiation."

I don't think we should talk about side channels ,outside of SDP, etc.

I also think it is important to point out that it is not a limitation of SDP - SDP allows you to indicate direction-specific capabilities using sendonly and recvonly. It is a limitation of "sendrecv" m- lines, and the old offer/answer assumption that you only need to indicate what you can receive.