VideoEncoderConfig.contentHint

[Djuffin]

WebKittens

@youennf

Title of the spec

VideoEncoderConfig's contentHint

URL to the spec

https://www.w3.org/TR/webcodecs/#dom-videoencoderconfig-contenthint

URL to the spec's repository

https://github.com/w3c/webcodecs

Issue Tracker URL

No response

Explainer URL

https://gist.github.com/Djuffin/c3742404b7c53ada227849c8b2b76b4c

TAG Design Review URL

No response

Mozilla standards-positions issue URL

No response

WebKit Bugzilla URL

No response

Radar URL

No response

Description

Problem and Motivation

Webcam content often require denoising and is often intelligible even when downscaled or with high quantization levels.
Screencast content of presentations or webpages with a lot of text content is completely unintelligible if the quantization
levels are too high or if the content is downscaled or otherwise blurry.

Video encoding libraries (like libvpx, OpenH264) provide a way for API users to inform the encoder about
type of the encoded content.
In some cases the web application can make a more-educated guess or take user input,
but currently there is no way to inform VideoEncoder what kind of content is being encoded.

Proposed solutions

Add a new field to VideoEncoderConfig

Let's add a contentHint field to VideoEncoderConfig. It will take video content hint values that are already used for MediaStreamTrack: "motion", "text", "detail".

This gives web developers a way to communicate to VideoEncoder the expected type of the content they intent to encode.
This setting is not intended to replace encoder-level settings completely but rather complement them
with a simpler hint that does not require broad knowledge of video encoders.

[marcoscaceres]

Discussed with colleagues internally. We think what is being proposed is suitable, provided the hint definitions are not too vague and it is clear how to use them to tune encoders.

Unless we hear otherwise over the next week, we are inclined to label this with "support".