determining common capabilities #569
Comments
Easy. From RFC 6184 section 8.1:
buff... |
Somehow related: #567 |
On Wed, Jun 22, 2016 at 12:34 PM, Philipp Hancke notifications@github.com
|
Do I have to accept the grim reality that there is no generic way to do this but I need to teach my JS all kinds of weird information about h264 and opus parameters? :-/ |
Yes. |
@fippo @robin-raymond With respect to codec-specific parameters, most settings fit into one of the following buckets: a. A boolean capability of the receiver that can be set as a parameter on the sender. Examples: useDtx and useInbandFec in Opus. If the receiver indicates useDtx = true, then it would prefer the sender to use DTX, so that useDtx = true can be set in sender parameters. Similarly, if the receiver indicates useInbandFec = true, then it would prefer the sender to use inband FEC, so that useInbandFec = true can be set in sender parameters. Note that useDtx and useInbandFec is not a sender capability because it is assumed that a sender will support it (this isn't always true, but if it isn't the sender should ignore the setting). b. A numeric capability of the receiver that can be set as a parameter on the sender. maxBitrate, maxFr and maxFs are examples of this. c. Weird cases. I think that profileLevelId may be one of these. As I read RFC 6184, when level-asymmetry = 0 or not provided (e.g. Chrome), the levels sent and received need to be the same. So in this case, you can take a receiver capability of profile-level-id=42e01f and set this on a sender (Edge) that has a capability of profile-level-id=42C02A but having Edge answer with profile-level-id=42C02A would not work because the levels need to be symmetric - you need to take the minimum of the capabilities and have Edge answer with profile-level-id=42e01f. Since with FF level-asymmetry=1, the situation may be different there (does FF tolerate an Answer of profile-level-id=42C02A?) |
@fippo Looking at the code, I would suggest the following approach to determining sender parameters (the intersection of the local Sender capabilities and the remote Receiver capabilities). RTCRtpCodecCapability. If the local codecs[i].name and codecs[i].clockRate match values within the remote capabilities, then we have a codec match, but need to figure out the values of other attributes for codecs[i]: a. preferredPayloadType: this is the remote codecs[i].preferredPayloadType value. b. ptime: this is set to the sender value, clamped by the receiver maxptime. c. numChannels: this is the smaller of the local and remote values. d. rtcpFeedback: this is the intersection of the supported values of the local sender and remote receiver for a given codec. |
I ran into another issue here when it came to RTX. The current code I have in adapter broken when it got something along the lines of:
Since I was merely checking that the left (local) and right (remote) code name matched I ended up putting rtx with the wrong apt in the common set (which didn't work). sigh |
Work-in-progress do not merge. Fix for Issue #569
Still outstanding: |
Trying to fix my remaining items for h264 interop between Chrome/Firefox and Edge I ran into an issue with determining the intersection of codec parameters and rtcpFeedback. It would be useful to at least have some high-level text about matching local and remote capabilities in the spec.
Some code lives here. The ORTC demo on Microsofts TestDrive also has a function filterCodecParams which does a similar thing.
After I determine that a codec is common (same name, clockrate and number of channels), I need to determine what rtcp feedback and parameters can be used.
rtcp-fb is easy, chrome sends this:
Since Edge only supports nack pli and goog-remb, only those two should be in the answer (I am not sure if there is a distinction between 'capabilities in the answer' and 'common capabilities').
This is also described in RFC 4585:
The parameters are something where I need advice. When Firefox offers, the profile-level-id is
whereas Edge has this capability
How do I deal with this? A filter operation as I do for rtcp-fb is not right.
Same problem for audio (so its not just a matter of h264 profile levels), Chrome sends
a=fmtp:111 minptime=10; useinbandfec=1
which currently gets passed into Edge's RTPSender.send (without issues, I assume it ignores them).
Should I rather put the 'left' (local) parameters into the answer and the offerer (browser engine) is responsible for figuring out the right thing to do?
This smells a lot like the offer-answer rules from RFC 3264:
the tl;dr of this is 'it depends' i guess
The text was updated successfully, but these errors were encountered: