[css-mediaqueries-5] Clarifications on [video-]dynamic-range MQs

[willcassella]

CSS Media Queries Level 5 introduced two new queries: dynamic-range and video-dynamic-range. Both have similar wording, and currently the wording for dynamic-range is like so:

dynamic-range represents the combination of max brightness, color depth, and contrast ratio that are supported by the UA and output device.

high
The combination of the User Agent and the output device fulfill all of the following criteria:

• it has a high peak brightness
• it has a high contrast ratio
• its color depth is greater than 24 bit or 8 bit per color component of RGB

standard

• One or more of the criteria for a high dynamic-range is not fulfilled.

The way Safari has implemented this media query is to make @media (dynamic-range: standard) checks always pass, and @media (dynamic-range: high) check against the capabilities of the current display device. This makes sense from a future-proofing perspective, as standard can be seen as a subset of high, and if ultra-high dynamic-range displays are ever introduced, high should be a subset of that.

However, the wording of the standard enum makes it sound as if standard and high should be mutually-exclusive. I think we should change this wording to reflect standard as a subset of high.

Additionally, the wording of "The combination of the User Agent and the output device..." has caused some confusion about what the capabilities of the user agent must be in order for the high query to pass. In the discussion surrounding the proposal for these queries, the idea was that this query would be a coarse representation of the display device's capabilities, and more detailed APIs such as MediaCapabilities could be used to gather specific information about the User Agent's capabilities for specific purposes. I think we should remove mention of the UA in the description of this media query to clarify this confusion.

[willcassella]

CC'ing participants in the original discussion for visibility: @tabatkins @gregwhitworth @astearns @svgeesus @Crissov @mwatson2 @AmeliaBR @chcunningham @vi-dot-cpp

[dbaron]

The other issue with the wording regarding "The combination of the User Agent and the output device..." is that User Agents may have different capabilities for CSS (and HTML) colors, colors in images, colors in canvas, colors in videos, etc. Which of these are required to be supported should probably be specified, both (separately) for dynamic-range and video-dynamic-range.

[nightpool]

As a developer, I agree that it would be kind of pointless to have (dynamic-range: high) return true if the actual user agent didn't support high dynamic range colors in at least some CSS-visible sense (CSS colors or colors in images). I think removing the mention of the UA in the description of this media query would mean that we end up speccing a property that is internally consistent, but isn't useful to developers and therefore doesn't gain adoption. And then in 12 months or so we'll be in the same place again and have to specify a new user-agent-video-dynamic-range media query.

[chcunningham]

In the discussion surrounding the proposal for these queries, the idea was that this query would be a coarse representation of the display device's capabilities, and more detailed APIs such as MediaCapabilities could be used to gather specific information about the User Agent's capabilities for specific purposes.

As a participant in the original discussion, I want to give a big +1 to ^this point. The dynamic-range MQ was motivated entirely by a need to describe the screen's capabilities. This is consistent with the overall purpose of Media Queries, which describe features (e.g. resolution) for the current media device (e.g. screen).

I think folks are confused by the video-dynamic-range feature. At a glance, this looks like it indicates when the UA is capable of playing HDR <video>, so it follows that the non-video dynamic-range would correspond to non-video things like CSS. This is understandable but definitely not the intent.

Both video-dynamic-range and dynamic-range describe the screen. The video-dynamic-range proposal concerns a weird quirk of user agents implemented in TVs. Given limited system resources and a clear priority use case, these devices have a quirky implementation where video and non-video are rendered in distinct "planes", which amounts to distinct screens with distinct capabilities. Generally speaking, the video plane has a much higher resolution and dynamic range than that of non-video. Traditional user-agents have only one plane, such that video-dynamic-range and dynamic-range should behave identically.

Again, video-dynamic-range just describes the screen. It does not describe support for HDR video playback. This is a much more complicated question (w/ parameters like codec, transfer function, metadata, ...) which Media Queries is ill-suited to answer (it is instead answered by navigator.mediaCapabilities.decodingInfo()). IMO, questions about canvas, CSS, images, etc, should similarly follow this model of offering separate feature detection, choosing whatever method is most idiomatic for their respective domains.

[svgeesus]

Both video-dynamic-range and dynamic-range describe the screen. The video-dynamic-range proposal concerns a weird quirk of user agents implemented in TVs. Given limited system resources and a clear priority use case, these devices have a quirky implementation where video and non-video are rendered in distinct "planes", which amounts to distinct screens with distinct capabilities.

Yes, exactly. In other words, for most devices, these will have the same value. Only for a TV with a high-res, HDR video buffer and a (lower res, non-HDR) "graphics overlay buffer" will they have different values.

So the specification needs to more clearly reflect that.

[css-meeting-bot]

The CSS Working Group just discussed [css-mediaqueries-5] Clarifications on [video-]dynamic-range MQs.

The full IRC log of that discussion

<fantasai> Topic: [css-mediaqueries-5] Clarifications on [video-]dynamic-range MQs
<fantasai> github: https://github.com//issues/6793
<fantasai> florian: We have 2 MQ which are similar, dynamic-range and video-dynamic-range
<fantasai> florian: 2 issues in 1 here
<fantasai> florian: one is primarily editorial, doing poor job of explaining the difference between these two queries
<fantasai> florian: notion of video plane is fairly rare concept that relates to TVs and how they have unusual screen buffer impls that we can query separately
<fantasai> florian: core question raised in this issue, besides clarification
<chris> +1 to that editorial clarification
<fantasai> florian: is that we have 2 values: high and standard
<fantasai> florian: You might assume, and this apparently is what Safari did, that any device will match standard and some will match high as well
<fantasai> florian: this is not how currently specced, though
<fantasai> florian: devices expected to match one or the other
<fantasai> florian: not standard and high simultaneously
<chris> q+
<fantasai> florian: The query about color-gamut does behave differently, it has 3 values
<fantasai> florian: and these are additional, when you match broader gamut you also match narrower one
<fantasai> florian: unclear what we should do here
<Rossen_> q?
<Rossen_> ack chris
<fantasai> chris: I agree, media capability does the same thing, of using supersets
<fantasai> chris: overall I think it's a better model, think we can easily make spec say that
<fantasai> chris: the reason to do it is that when we add super-high later on, we want it to be a superset that still passes high
<fantasai> chris: so I think we should change the spec; not because it's what implemented, but because it makes more sense
<fantasai> dbaron: Wanted to add, think there's a 3rd issue
<fantasai> dbaron: currently wording in spec about combo of UA and output device
<fantasai> dbaron: needs clarification what the UA part of it means
<chris> see also https://w3c.github.io/media-capabilities/#hdrmetadatatype
<fantasai> dbaron: definition of high says, combination of UA and output device fulfill all the following criteria
<fantasai> dbaron: there are a number of ppl in the issue saying that the UA bit should be ignored, and should only be query about the output device
<Rossen_> ack dbaron
<fantasai> dbaron: but if this is question of UA capability, which are you considering if UA can do for some things but not others
<florian> q?
<fantasai> dbaron: e.g. videos but not images, or ...
<chris> q+ to also mention HDR is opt-in and can change dynamically
<fantasai> florian: I suspect querying device and not UA is useless
<fantasai> florian: because if the UA isn't able to access the capability, it's not helping you
<fantasai> florian: but if capability varies within UA, that's a problem
<fantasai> Rossen_: Is one a superset of other? Is UA superset of device?
<fantasai> dbaron: I wouldn't think of it that way
<fantasai> florian: We're querying union of both
<fantasai> florian: need both UA and device to be capabe
<fantasai> chris: Also, HDR capability can vary over time
<fantasai> chris: high power usage
<fantasai> chris: so off by efault
<fantasai> chris: so need to know that it can change
<fantasai> chris: One, is the device capable of HDR mode, and then two, is it in that mode?
<Rossen_> s/Is UA superset/Is UA subset/
<fantasai> chris: we are mixing those two up
<fantasai> florian: we do
<Rossen_> ack chris
<Zakim> chris, you wanted to also mention HDR is opt-in and can change dynamically
<Rossen_> ack fantasai
<drott> fantasai: we should take a resolution to change the query to superset model
<drott> fantasai: multiple things to be clarified
<drott> fantasai: let's ask editor to go back to the thread to work out results
<drott> fantasai: a lot of the clarifications are straightforward
<drott> fantasai: need a resolution to change the media queries
<fantasai> florian: Don't quite agree, question of capability or in HDR mode currently
<fantasai> florian: or videos vs images etc.
<fantasai> florian: not quite clarification, is change in functionality
<fantasai> florian: puzzle how to answer if we don't change the syntax
<chris> q+
<fantasai> florian: so can do things fantasai highlighted, but not enough to solve the issue
<Rossen_> ack chris
<fantasai> chris: media capabilities spec talks about capability, "can the device do it"
<fantasai> chris: if we clarify that way, need the other query as well
<fantasai> chris: the capability and concrete "am I in this mode" are separate
<fantasai> florian: Is it expected to remain this way?
<fantasai> chris: especially for mobile, definite power drained
<fantasai> chris: can be switched automatically, doesn't require user action
<fantasai> chris: but there needs to be a capability
<fantasai> chris: and need to know which mode you're in
<drott> fantasai: i think we should spec as being able to use the switch
<drott> fantasai: from a media queries point of view, how are we styling the document - many of those questions on depend on current mode
<drott> fantasai: [missed parts]
<fantasai> florian: what do we do about if UA can do for images and video but not CSS colors?
<fantasai> florian: or some other variation?
<fantasai> florian: should we consider the UA does or does not fulfill the criteria?
<fantasai> florian: or do we think about query in a different way
<fantasai> florian: I haven't thought about deeply, but maybe you would have high-dynamic-range: video | canvas | etc.
<fantasai> florian: and would get something true if that specific thing is in HDR mode
<fantasai> Rossen_: sounds like a convesation to continue in GH
<fantasai> Rossen_: and doesn't seem we're ready to resolve just yet
<fantasai> Rossen_: propose we stop here, and then after working out proposal in issue, bring back and we'll record a resolution
<fantasai> Rossen_: but hopefully attracted some help on this issue

[frivoal]

Having had a little side discussion with the chrome folks, the tentative conclusion is that the way Safari implemented this is what we'd like the spec to align to. Specifically:

• standard is a subset of high, in the same vein as the color-gammut media feature values.
• Even though we just have two values, we do want to keep it as "standard" and "high", not turn it into "none" and something else so that it'd work in a boolean context, because in the future we may need to add one or more values to distinguish between high and even higher, depending on what sort of device gets made.
• This is about HDR capability, not about "is HDR active". The later question could possibly be interesting, but it's a more complicated one as that means different things on different types of devices, and the primary use of the MQ is just to gate-keep you entire HDR logic
• there is no author desire for sub-dividing the querry between hdr-for-video, hdr-for-images, hdr-for-css, etc, because while that would be more fine grained, it wouldn't actually be fined grained enough to answer all your hdr questions. You might still need to wonder about image/video formats, contrast ratios, quality of tone-mapping…… So it's not really a good idea to make the MQ more complex if it's not actually going to answer the nuances, and having it as a blunt question to know if you should bother with HDR at all seems to be what authors are actually calling for.
• There's nothing unusual about this querry when it comes to testing the capabilities of the output device vs that of the user agent vs that of the combination, so we should neither put special emphasis on it, nor go out of our way to claim that we're only testing one or the other.

In other words, a hypothetical different design, like

hdr:
    none | 
    [ 
        [img | img(<img-format>)] |
        [video | video(<video-format>)] |
        css |
        canvas
    ] &&
    active?

is not actually appealing to authors, and would add complexity for no practical reason, thus we should spec what Safari is shipping.

[frivoal]

Misconfigured the log bot while discussing this. The discussion is at https://logs.csswg.org/irc.w3.org/css/2021-12-15/#e1452098, and the resolution was :

RESOLVED: dynamic-range: standard also matches when dynamic-range: high matches