Initial support for AV1-SVC Dependency Descriptor

[lminiero]

As the title says, this is an initial effort to start tinkering with AV1-SVC. Specifically, this patch adds support for the new Dependency Descriptor RTP extension, which is needed to get SVC working. We don't do anything with the extension right now, we just negotiate it and copy it as a buffer so that it's available to plugins (and plugins can pass it along when relaying packets).

It's a draft for a few different reasons. The most important one is that I can't really test it, right now: any time I launch Chrome with the flag to negotiate the DD extension, if I enable SVC the extension seems to be broken (always too small), and so when relayed it causes broken video. I suspect it's an issue with the implementation of the extension in Chrome for Linux, but I can't know for sure until you people give it a try. The other reason is that I'm still working on the parser for the (awful) format of the extension: thanks to the incredibly kind @Murillo, I have access to a .pcap with extensions like they should be, and so I'm not at a good point parsing the content, but I'll need a working browser to test this in the "wild", e.g., to decide when to drop layers and when to pass them along.

If you want to test what's in there right now, you'll first of all need to launch Chrome with a custom flag to enable the DD extension:

--force-fieldtrials=WebRTC-DependencyDescriptorAdvertised/Enabled/

and then enable the "Experimental web platform features" flag to add AV1 scalability modes:

chrome://flags/#enable-experimental-web-platform-features

To make sure AV1-SVC is enabled in your browser, check the capabilities using the JavaScript console:

RTCRtpSender.getCapabilities('video');

This should list several scalabilityModes for "video/AV1X" in the codecs list, and "https://aomediacodec.github.io/av1-rtp-spec/#dependency-descriptor-rtp-header-extension" should be listed in the headerExtensions instead.

Once you're sure your browser should be ok, you can launch the EchoTest by passing a couple of query string parameters to force AV1 and a specific scalability mode, e.g.:

http://localhost:8000/echotest.html?vcodec=av1&svc=L3T3

If everything works, you should see the remote video as well: there's no buttons to choose layers yet since, as I anticipated, we currently do nothing with the extension if not pass it along. In my case, remote video doesn't work as the extension is always too small (e.g., ~8 bytes), while it should be much larger when it contains all the template definitions (~95 bytes in the example Sergio sent me). If it works for you, an unencrypted .pcap file or an .mjr file for the session would be quite appreciated, as it would allow me to try and replicate a real-time scenario while I figure out what's wrong in my browser.

[lminiero]

PS: the same query string parameters work for the VideoRoom demo as well (assuming you created a room with AV1 support), even though I don't see much point in testing that yet, considering SVC is not really used at this stage.

[lminiero]

PPS: I configured the legacy demo Janus instance to use this branch, so if you have an AV1-SVC compliant browser you can also test it online without installing this branch:

https://janus-legacy.conf.meetecho.com/echotest.html?vcodec=av1&svc=L3T3

[lminiero]

Note: I just edited the PR description to clarify you need to enable the enable-experimental-web-platform-features flag in Chrome too, or scalability modes for AV1 won't be available.

[danjenkins]

Unfortunately I don't get any video when Scalability Modes is enabled

Chrome Canary 94.0.4584.0 MacOS Intel

[lminiero]

Found out the problem is in Janus, since we're rejecting the a=extmap-allow-mixed attribute, which is needed for the DD RTP extension (which uses two-bytes extensions since they can be larger than 16 bytes). Just negotiating the attribute isn't enough, since our RTP extension parser and builder currently only supports one-byte extensions. I'll work on that and update the PR accordingly.

[lminiero]

This new commit implements support for two-byte header extensions, and does make video work for me finally. This means we now have a good foundation for building some SVC logic as the next step (sooner or later 😉 )

[danjenkins]

I can confirm I also get working video at the above url with Scalable Video and the field trial enabled

[lminiero]

Just as I've done with #2685, I've decided to limit the scope of this PR, in order to simplify and streamline its main purpose, and leave the rest of the work to future effort. Specifically, since this PR was on hold for quite some time because of issues in the SVC selection process (i.e., how to effectively only forward some layers and not all of them), I removed the test logic I had in the EchoTest, where I only left the ability to negotiate the new Dependency Descriptor extension: together with the ability to capture and encode an SVC stream in the demo, this should provide a useful playground for future testing. I also removed the DD support I had added to the VideoRoom, as it would be useless at the moment (we don't use the SVC features in the SFU).

That said, the PR as it is is still very important, not only because it adds support for negotiating, parsing and forwarding the new DD extension, but also because in order to do that we had to implement support for Two-Byte header RTP extensions, which was missing in Janus and we now have (and will likely be useful for other things in the future).

Once we have cycles for that, we'll get back to actually use the SVC information we can now parse: last time I checked, the main issue was that, after a specific layer is selected, the whole DD must be rewritten (it cannot be forwarded as it is, since the active streams must be updated), which will be a huge pain due to the awful format of the DD itself.

[SetoKaiba]

I see no scalabilityModes with Chrome 113.0.5668.0 dev. Any idea?