Distinguish video rendering and screen support for color gamut

[mwatson2]

Video renderer support for wide color gamuts (P3, Rec.2020) is distinct from screen capability and may be dependent on video codec.

No existing screens support all of Rec.2020 (the primaries are on the spectral locus, so you need lasers!).

It should be possible to query whether rendering of these colorspaces is supported with a given video codec (i.e. whether the renderer has the ability to map from that colorspace to the display).

It's not clear what information it would be possible to get about the actual screen capability or what application there could be for that. Display manufacturers typically consider precise definition of the display colorspace (i.e. the coordinates of their primaries) as highly proprietary information. Typically, one might assume that if the device manufacturer has gone to the trouble of supporting interpretation / rendering of WCG content, then it is worth supplying that content.

Note that there will be devices where display and video renderer support WCG, but the graphics renderer does not.

[PauKerr]

I suggest that the VideoConfiguration dictionary be extended with a non-required element to allow the user to specify the rendering capability needed. We have discussed that the video and graphics (as represented by CSS) capabilities can differ, especially when there is dedicated hardware support for video playback. If we defer covering this issue in the definition of this API, we still need to be able to query for both decode ability and render ability to truly determine whether it is worth streaming HDR content. Something like this:

`
dictionary VideoConfiguration {
required DOMString contentType;
required unsigned long width;
required unsigned long height;
required unsigned long bitrate;
required double framerate;
RenderCapability renderCapability;
}

dictionary RenderCapability {
DOMString colorProfile = “sRGB”;
unsigned long colorDepth = 24;
DOMString transferFunction;
}
`
Here colorProfile elements would be the same as those proposed in the CSSOM extensions: rec2020, sRGB, etc. with colorDepth no longer be limited to 24 and transferFunction covers the issue raised in #10

While the user agent cannot guarantee the render ability of, for example, an attached monitor, it can determine the capability of the HDMI connection and minimally send the appropriate bitstream to the connected device.

[chcunningham]

Let me add some context for how I've been thinking about this so far (ideas not quite fully formed).

For display capabilities, current thinking is we should signal wide gamut via the pixelDepth/colorDepth CSS property. Mounir has recently implemented signalling higher pixel depths.

For decode and rendering capabilities, my thinking is to lump these together when it comes to signalling isSupported. For instance, today chrome can always decode HDR content, but (for now) it requires special flags to put the rendering engine in a mode that manages the colors correctly. As it stands, we are only reporting decode support for HDR vp9 content when chrome's color management is enabled. I had not considered additionally checking on the HDMI connection. Need to think on this some more.

For how to query rendering capabilities, I think the RenderCapability proposal is a reasonable route. For VP9, hot off the presses is a new proposal for the codec string that actually contains all of this information. Chrome will ship parsing for this string soon. I don't know for HEVC if the codec string might also be used to signal these things. If they don't we may definitely need something like RenderCapability.

[mwatson2]

The HEVC codec string does not include rendering information like this. Strictly, it's not a property of the codec, which just decides the compressed data to uncompressed form, without needing to know how the uncompressed data is supposed to be rendered.

[mounirlamouri]

I'm in general not in favour in leaking the video/graphics rendering capabilities that TV have into this API this way. As mentioned in #25, this is a larger issue and trying to add something here and there is unlikely the right approach. In a regular browser context (desktop/laptop/phone), we think that using the CSS information should be enough. The fact that this isn't enough on some hardware should be resolved in #25 IMO. One approach for example would be to expose the "video layer" capabilities for these devices in some ways. Would that solve your issue?

[PauKerr]

My intent in extending the VideoConfiguration to represent rendering capabilities is not to discover what an HDMI attached device is capable of rendering but is to determine the current capabilities of the platform. The main use case would be to determine the render capability of the display embedded on the browser host platform: for example, the laptop screen.

For the case where a display that is connected via HDMI, and the host platform is able to send video rendered to the attributes specified over the HDMI link because the currently attached device will allow the HDMI mode to be switched to support those attributes, then it should simply respond with MediaCapabilitiesInfo::supported since it can output the signal but has not real confirmation that the attached unit will display it as specified.

Desktop browsers will exist on platforms with HDR video support well ahead of a general css-based solution. This capability query will allow for the core use case to move forward.

[mwatson2]

This issue is addressed by the current proposal in PR #124. I propose we close this issue.

[chcunningham]

These things were removed from the PR, but this issue more or less covered by #25 and #135. Happy to reopen if you think I've overlooked anything.