Advanced use cases for WebCodecs

[tidoust]

Other issues deal with more specific needs for WebCodecs. More generally speaking, what are the use cases that should be given priority in WebCodecs for media production?

Raised in:

• Hello WebCodecs
• WebCodecs VideoEncoderConfig
• Memory access patterns in Web Codecs

[chcunningham]

The WC use cases I've collected so far are things like:

1. decode and render very specific frames of video (e.g. on timeline or in larger player viewport as users seek around the file)
2. modifying VideoFrame pixel data (e.g. background replacement, watermarking)
3. exporting finished productions to encoded files
4. client-side transmuxing

From that list

1. Is well supported today (see Hello WebCodecs).
2. Is well supported, but may involve more copies than we'd like (see Memory access patterns in Web Codecs). Some paths like WebGL are already well optimized.
3. Is supported, but I expect more encoder settings are desired (see WebCodecs VideoEncoderConfig).
4. Same as 3.

I look forward to hearing more use cases from folks as well as hearing which of the gaps identified above are most pressing!