Use the same codec names as the media APIs.

[pthatcherg]

MediaCapabilities (and other media APIs) uses a content type of "video/foo; codecs=A.B.C". We should probably use the "A.B.C" instead of what we have currently. For example, instead of "h264", use "avc1.4200" or instead of "vp9", "vp09.00.10.08". It's a little crazy and I can't find any RFCs for these things, but here is where Chromium parses them:

https://cs.chromium.org/chromium/src/media/base/mime_util_internal.cc?g=0&l=820

[markafoltz]

Is this what you are looking for? It covers some, but not all of the MPEG codecs.

https://tools.ietf.org/html/rfc6381

There may be an equivalent RFC for the VP*/AV1 codecs.

The media APIs don't make any explicit reference to any RFCs. I have no idea what implementers are supposed to do, other than copy each other's source code :-P

[markafoltz]

Spoke with Dan Sanders, there are other RFCs for the webm container we can refer to as well.

He noted that enumeration of codec strings for any given codec family is a hard problem because of the incredibly rich syntax for codecs - so we should avoid that in our protocol design.

[markafoltz]

Take a look at these specs for VPX and AV1 and you'll see what I mean:

https://storage.googleapis.com/downloads.webmproject.org/docs/vp9/vp-codec-iso-media-file-format-binding-20160516-draft.pdf

https://aomediacodec.github.io/av1-isobmff/#codecsparam

[markafoltz]

It sounds like there wasn't an explicit action recorded in the last F2F minutes [1], but for the record, we discussed consulting with the Media WG folks working on Media Capabilities and related APIs to get feedback on this issue.

[1] https://www.w3.org/2021/02/23-webscreens-minutes.html#t02

[chrisn]

The minutes from the joint meeting with Media WG are here. Summary:

• We can use references in the WebCodecs registry for codec string details: https://www.w3.org/TR/webcodecs-codec-registry/

[markafoltz]

Want to discuss briefly and propose putting up a pull request to update the specification to explicitly use the WebCodecs registry to resolve this issue.

[markafoltz]

@chrisn, at the F2F [*] you mentioned you would follow up with any thoughts on how to extend the codecs name list beyond what is defined through WebCodecs. Maybe we can just fall back on the language that is already there, which is:

This must be a single-codec RFC 6381 codecs parameter

for codecs not listed in WebCodecs.

[*] https://www.w3.org/2022/06/08-webscreens-minutes.html#t08

[chrisn]

Yes, that could work, but it doesn't really help to know how to implement. Is a fall back enough? If we do need to be more specific, then we'd need to find relevant specs which aren't in the WebCodecs registry, e.g., via MP4RA - and perhaps turn that into its own spec, which as you noted earlier is a hard problem.