WebM AV1 video codec

[AshleyScirra]

Description

Note

This proposal is split from #425 which originally proposed video and audio together. This proposal is only for video. Since video normally also includes audio, it probably depends on #484 (the audio codec proposal for Opus).

For many years there has not been a single video codec that can be relied upon to work in all browsers. Further, proprietary codecs can involve licensing fees that make some uses cases uneconomical, including making it difficult for open source projects to support. This is a major point of interop pain, resulting in having to transcode media files to multiple formats, or use workarounds like codecs implemented in WebAssembly.

I would like to propose that browsers commit to supporting the AV1 video codec and the WebM container. AV1 is a well-specified modern, open, royalty-free codec, and both AV1 and WebM already have substantial support across the industry. If all browsers consistently supported AV1 across all relevant web media features (video element, WebCodecs, WebRTC, MediaRecorder) then there would finally be one codec that can be relied upon cross-browser for video support.

As it stands, AV1 is not yet fully supported in Safari (it appears to be supported with Safari 17 but only on some devices), and Edge requires the "AV1 Video Extension" to be installed from the Microsoft Store so it can't be relied upon to work out-of-the-box.

According to caniuse, all major browsers already have support for WebM as a container format (barring bugs).

Specification

AV1 is specified by AOMedia. However AFAIK no web-related specification currently mandates support for any specific codecs.

Open Issues

No response

Tests

It looks like WPT only has tests specifically mentioning AV1 for WebCodecs and one AV1 test for WebRTC. Perhaps there should be new tests added which make use of decoding AV1 video in various scenarios with all relevant APIs to ensure robust cross-browser support.

Current Implementations

• Blink
• Gecko
• WebKit

Standards Positions

Chrome and Firefox already support AV1. Edge already has optional support for AV1. WebKit supports AV1 and has started shipping support in Safari 17. Further it seems there's a case to update the specification to mandate AV1 support.

All browsers already have some support for WebM.

Browser bug reports

Not sure any public issue exists for Safari support for AV1.

Developer discussions

This has been an important and widely-discussed topic since the introduction of the video element in the late 2000s. Apparently it was originally proposed to require support for Ogg Theora/Vorbis, but that was replaced with wording along the lines of "there are no known codecs that satisfy all the current players". I am hopeful that the situation has now changed, with increasing support for AV1, and all major browser makers are governing members of the Alliance for Open Media which specifies AV1.

Ideally support for AV1 could be mandated in the specification, presumably also with WebM as the container. I don't see why not if all browsers have added some degree of support already, and it would give additional confidence to web developers that the codec can be relied upon.

Polls & Surveys

No response

Existing Usage

No response

Workarounds

Transcoding media to multiple formats, or using WebAssembly for codecs. Both have significant downsides (e.g. complexity, cost, performance).

Accessibility Impact

Users using open-source software may find that media on the web is unavailable if it depends on proprietary codecs that it is infeasible for the open-source project to support.

Privacy Impact

No response

Other

No response

[jzern]

For reference, here are the format bindings for AV1:

• MKV/WebM
• MP4

[jzern]

cc @robUx4

[bdougherty]

I would also like to see this added to Interop 2024, but with MPEG-4 container support as well.

[othermaciej]

If this is taken up, the requirement should be AV1 in an ISOBMFF (MP4) container, as that is a standard while WebM is not. I do not believe the ISO container format has the kinds of royalty-bearing patent issues as other MPEG family standards, so it would also meet the stated need of this issue.

(I take no stand at this time on whether this should be included in Interop 2024 or not).

[o-t-w]

@AshleyScirra Was there a reason why you specifically ask for WebM over MP4? I'd love to see universal support for AV1 but of MP4 seems like the more ubiquitous container.

[foolip]

Thank you for proposing WebM AV1 video codec for inclusion in Interop 2024.

We wanted to let you know that this proposal was not selected to be part of Interop this year.

This is because we got many more proposals than we could include in this year's project. Note that individual vendors may nevertheless choose to advance work in this area during the forthcoming year. We would welcome this proposal being resubmitted again next year if necessary.

For an overview of our process, see proposal selection. Thank you again for contributing to Interop 2024!

Posted on behalf of the Interop team.