User Preference Media Features Client Hints Headers

[tomayac]

Ya ya yawm TAG!

I'm requesting a TAG review of User Preference Media Features Client Hints Headers, a set of
HTTP Client Hints headers around user preference media features as defined by Media Queries Level 5.

• Explainer¹ (minimally containing user needs and example code): https://github.com/WICG/user-preference-media-features-headers/blob/main/README.md
• Spec (draft): https://tomayac.github.io/user-preference-media-features-headers/
• Security and Privacy self-review²: https://github.com/tomayac/user-preference-media-features-headers/blob/main/security-privacy-questionnaire.md
• GitHub repo (if you prefer feedback filed there): https://github.com/tomayac/user-preference-media-features-headers
• Primary contacts (and their relationship to the specification):
  • Thomas Steiner (@tomayac), Google, spec author
  • Yoav Weiss (@yoavweiss), Google, spec buddy
  • François Beaufort (@beaufortfrancois), Google, Chromium implementer
• Organization/project driving the design: Google
• External status/issue trackers for this feature (publicly visible, e.g. Chrome Status): https://chromestatus.com/feature/5642300464037888

Further details:

• I have reviewed the TAG's Web Platform Design Principles
• The group where the incubation/design work on this is being done (or is intended to be done in the future): WICG (in the future)
• The group where standardization of this work is intended to be done ("unknown" if not known): unknown
• Existing major pieces of multi-stakeholder review or discussion of this design: [css-mediaqueries] Expose User Preference Media Queries as HTTP Client Hint or HTTP Header w3c/csswg-drafts#4162
• Major unresolved issues with or opposition to this design: Should Sec-CH-Prefers-Reduced-Data be defined for consistency, or yield to the existing Save-Data header?
• This work is being funded by: Google

You should also know that...

This is a feature we (Google) are hoping to use for Google Search (google.com).

We'd prefer the TAG provide feedback as (please delete all but the desired option):

🐛 open issues in our GitHub repo for each point of feedback

[tomayac]

More related links:

• WebKit thread: https://lists.webkit.org/pipermail/webkit-dev/2021-May/031856.html
• Mozilla Standards Position: User Preference Media Features Client Hints Headers mozilla/standards-positions#526

[beaufortfrancois]

Following @kenchris' suggestion, this is a gentle ping as we'd like to move forward this tiny change ;)

[tomayac]

@beaufortfrancois has landed an implementation (👏) of Sec-CH-Prefers-Color-Scheme in Chrome Canary (≥92.0.4514.0).

We have made a simple demo available that showcases the use case we want to enable.

The explainer now includes static screenshots of the demo's output in case you don't feel comfortable trying it in Canary.

[beaufortfrancois]

@torgo I see that a design principe issue has been filed at w3ctag/design-principles#307 following the TAG teleconference review last week.
Is there a way to move forward with this without blocking on the design principle issue?

[atanassov]

@beaufortfrancois the design principle issue you referenced is a general one that we opened and not one that blocks this particular design review. In fact, the motivation and proposed solution makes sense from performance benefits point of view and we are continue to work through the review.

One concern I want to explore further is about the fingerprinting capabilities that the feature enables. A combination of such media feature hints can be used to allow much easier evasion for phishing or other ill-intended sites. Have you considered such scenarios?

[tomayac]

One concern I want to explore further is about the fingerprinting capabilities that the feature enables. A combination of such media feature hints can be used to allow much easier evasion for phishing or other ill-intended sites. Have you considered such scenarios?

Just wanted to note that no additional information is exposed. The same information is exposed by the user preference media features from CSS, paired with JS:

if (matchMedia('(prefers-color-scheme: dark)').matches) {
  // Send info to server that the user prefers dark mode.
}

[atanassov]

Just wanted to note that no additional information is exposed. The same information is exposed by the user preference media features from CSS, paired with JS:

That's a given. My point was that with this feature an evasion can occur without sending any bytes to the client.

[beaufortfrancois]

Technically, bytes are sent to the client during the Critical Client Hint exchange. That being said, I’m not sure how not sending bytes relates to the threat model. Can you explain what you mean by “evasion for phishing” and how sending the information in request headers would help malicious servers there? A concrete example would be helpful.

[beaufortfrancois]

@atanassov (gentle ping)

For info, here's the blink-dev@ intent to ship thread.

[tomayac]

Dear TAG,

Thanks for your reactions so far! We have looped in our internal Chrome security team to discuss @atanassov's remark, but they found no new issues and approved of us proceeding.

The Blink API owners would ideally wait to hear any last concerns from this group before (hopefully) begin LGTM'ing the Intent to Ship.

Thanks,
Tom

[tomayac]

FYI: this feature is shipping in Chrome 93. We've (again) pinged WebKit and Mozilla for their positions:

• WebKit thread: https://lists.webkit.org/pipermail/webkit-dev/2021-May/031856.html
• Mozilla Standards Position: User Preference Media Features Client Hints Headers mozilla/standards-positions#526

[tomayac]

It would make most sense for the CSS WG to own this longterm I think.

[atanassov]

@plinss, @cynthia and myself did another review of this during our Gethen vf2f. First of all, thank you for following up on with your security team and getting an LGTM from them. I personally won't push on this further given it's already captured.

Another issue with this proposal is the affect it has to existing styling patterns. All conditional styling today takes place on the client as a result of style recalc in combination of various environment changes - color scheme, reduced motion, transparency etc. Exposing this through client hints will promote such conditional handling on the server side which will arguably negate some of the perf wins described and also require that state is kept on the server side. Ex. UA navigates to a.com in light mode exposed in the client hints. The server provides a light version of style.css and appropriate other content. The mode changes to dark in the same session - the server needs to recognize the change and this implies the client needs to reload possobly all the resources.

Finally, I don't believe CSSWG is the appropriate venue for this work. It is the right place to define media capabilities, not client hints.

[tomayac]

@plinss, @cynthia and myself did another review of this during our Gethen vf2f. First of all, thank you for following up on with your security team and getting an LGTM from them. I personally won't push on this further given it's already captured.

Thanks for the review and the always helpful comments!

Another issue with this proposal is the affect it has to existing styling patterns. All conditional styling today takes place on the client as a result of style recalc in combination of various environment changes - color scheme, reduced motion, transparency etc. Exposing this through client hints will promote such conditional handling on the server side which will arguably negate some of the perf wins described and also require that state is kept on the server side. Ex. UA navigates to a.com in light mode exposed in the client hints. The server provides a light version of style.css and appropriate other content. The mode changes to dark in the same session - the server needs to recognize the change and this implies the client needs to reload possibly all the resources.

This proposal solves the first request problem. The user changing themes afterward is always expected to require new resources. These could have been pre-loaded proactively during idle time of the page, or be fetched lazily on-demand. It does not interfere with client-side handling at all, for example in patterns like the following:

<link rel="stylesheet" href="dark.css" media="(prefers-color-scheme: dark)" />
<link rel="stylesheet" href="light.css" media="(prefers-color-scheme: light)" />

For performance reasons, the server could have inlined dark.css based on Sec-CH-Prefers-Color-Scheme: dark, and then only add the line <link rel="stylesheet" href="light.css" media="(prefers-color-scheme: light)" /> in the server-generated source code, so loading of light.css would work on-demand.

Finally, I don't believe CSSWG is the appropriate venue for this work. It is the right place to define media capabilities, not client hints.

Understood. We'll evaluate other options then. Device-Memory is edited by the Web Performance WG, which, @yoavweiss, might be an appropriate venue?!

[yoavweiss]

Typically, Client Hints were added as part of the specifications that define the client side APIs, so it seems to me that CSS would've been a good venue. But barring that, it may be interesting to discuss this in the WebPerfWG and see what that discussion leads to.

[atanassov]

For performance reasons, the server could have inlined dark.css based on Sec-CH-Prefers-Color-Scheme: dark, and then only add the line <link rel="stylesheet" href="light.css" media="(prefers-color-scheme: light)" /> in the server-generated source code, so loading of light.css would work on-demand.

Thank you for the great example @tomayac. I really like it because it is a good illustration of how the proposed micro-optimization to load performance could be achieved. At the same time it demonstrates my concern of shifting client-side style and resource management to the server.

In this case, if the dark.css is inlined into the document as a number of CSS rules. When the color scheme changes and light.css is loaded, the user will experience unexpected effects because rules from the two stylesheets will co-exist (because one of the stylesheets is inlined).

It is hard to justify such a huge shift of client-to-server responsibility and all the potential breakage for arguably minimal performance gain. At least in the case of media features, not CH in general.

[tomayac]

In this case, if the dark.css is inlined into the document as a number of CSS rules. When the color scheme changes and light.css is loaded, the user will experience unexpected effects because rules from the two stylesheets will co-exist (because one of the stylesheets is inlined).

In this case, you'd simply programmatically turn the inlined stylesheet off through its media attribute:

<style media="not all">
  /* Whatever is here, it's not applied anymore. */
</style>

[beaufortfrancois]

I've created a naive demo that shows how this can be done quite easily. You can play with it at https://bloom-accessible-offer.glitch.me/ and see code at https://glitch.com/edit/#!/bloom-accessible-offer?path=server.js

TLDR;

    <style media="(prefers-color-scheme: ${prefersColorScheme})">
      body {
        color: ${prefersColorScheme === 'dark' ? 'white' : 'black'}
      }
    </style>

    <link rel="stylesheet" href="${prefersColorScheme === 'dark' ? 'light' : 'dark'}.css" 
        media="(prefers-color-scheme: ${prefersColorScheme === 'dark' ? 'light' : 'dark'})" />

[plinss]

@tomayac @beaufortfrancois I'm fairly sure the current co-chair of the CSSWG is well aware of how to apply media queries to a stylesheet (as is this former co-chair of the CSSWG).

The point we're trying to make is: while it's certainly possible to use media feature client hints in a way that doesn't break user experience, there are many ways that it can be used that can. And those ways seems to be more likely to be employed by those without a deep understanding of what they're doing.

This feature seems like a giant foot-gun that only serves to shave a few bytes off the wire. We're questioning if the gains outweigh the risks.

[tomayac]

@tomayac @beaufortfrancois I'm fairly sure the current co-chair of the CSSWG is well aware of how to apply media queries to a stylesheet (as is this former co-chair of the CSSWG).

No doubt you all are aware, it was purely in response to "the user will experience unexpected effects because rules from the two stylesheets will co-exist".

The point we're trying to make is: while it's certainly possible to use media feature client hints in a way that doesn't break user experience, there are many ways that it can be used that can. And those ways seems to be more likely to be employed by those without a deep understanding of what they're doing.

Not sure where the argument is headed; are you saying someone knowledgable enough to work with HTTP headers would then at the same time not be knowledgable enough to work with CSS, as in the back-end/front-end divide? If so, yes, this might be the case. In our communication we have positioned the feature as aimed at sites of the scale of google.com, but definitely not the average website.

This feature seems like a giant foot-gun that only serves to shave a few bytes off the wire. We're questioning if the gains outweigh the risks.

On the Google Search team we were trying to make the case that it's not just "a few bytes", plus that for high traffic volume sites like google.com every byte does make a difference. Again, this is not aimed at the average website. Maybe we should make this even clearer!?

[plinss]

@tomayac @beaufortfrancois are you available to join us for a brief conversation about this? I suggest November 22 at 17:30 UTC (9:30am PST)

[tomayac]

Hello @plinss! The above date and time would work for me.

[plinss]

Thanks @tomayac, please join us on https://whereby.com/w3ctag

[tomayac]

Thanks for having me at the TAG meeting yesterday. I have contacted @pbergstr internally regarding the data request, but he's out for the Thanksgiving break, so I suppose very little will move until after the holidays… 🦃

[plinss]

Thanks for joining us @tomayac.

One follow-up question: we were wondering how this approach compares to the use of H/2 or H/3 server push?

e.g. the HTML links to two stylesheets with appropriate media queries, when serving the original HTML request the server starts pushing both stylesheets. When the client parses the links to the stylesheets, it sees that one isn't necessary right now, so cancels the push of that stylesheet and instead adds it to a low-priority queue to fetch later.

While that approach will likely send a few packets of an unnecessary stylesheet, it also avoids the delay/round trip of the client hints as well as the additional header payload, so may lead to overall similar performance (if not better depending on the placement of the stylesheet links in the HTML). We'd love to see data comparing the two approaches in ideal cases.

[yoavweiss]

One follow-up question: we were wondering how this approach compares to the use of H/2 or H/3 server push?

Seems worthwhile to point out that some browsers never shipped H3 server push, and intend to remove the H2 variant at some point. (potentially with Early Hints as a replacement)

e.g. the HTML links to two stylesheets with appropriate media queries, when serving the original HTML request the server starts pushing both stylesheets. When the client parses the links to the stylesheets, it sees that one isn't necessary right now, so cancels the push of that stylesheet and instead adds it to a low-priority queue to fetch later.

The network stack (which is responsible for server push) has no notion of media queries, so doesn't have the info required to cancel that push, even if canceling pushes was a thing that actually works. (It's doesn't, because of the extra RTT it'd take the cancel frame to reach the server, as well as buffer bloat).

In practice, this would result in both CSS files being downloaded, with the unneeded CSS likely being a blocker to more important content.

While that approach will likely send a few packets of an unnecessary stylesheet, it also avoids the delay/round trip of the client hints

Note that Client Hints reliability fixes the delay/round trip issue that Client Hints had with critical HTML content negotiation.

[atanassov]

@plinss, @torgo and myself had one more extended review of this entire proposal during our Madripoor vf2f.

First of all, thank you for bearing with us while we've been working on this review. Unlike the UACH base feature which we support, this proposal has problems we can't endorse as a general mechanism of the Web platform.

As you mentioned - ...this is not aimed at the average website.. Not all Web platform features are aimed or used by all websites. However, Web features must be predictable and possible for use by anyone, safely.

The base problem of this proposal is the exposure of user state rather than client capabilities of user preference media features. This allows user state handling on the server side which can lead to race conditions and undesired error state and user experience. One that doesn't exist today with media features which are evaluated and handled on the client side. This is not a safe pattern and should be avoided.

During our call we discussed multi stakeholder interest. You mentioned of few other large Web properties that would be interested and willing to use the feature for similar benefits to those of google.com. That isn't surprising. They employ enough engineers and will be able to take the hit of implementation and support. What we don't see is evidence of other browser vendors interested to support the feature.

We don't see evidence or use cases of user preference media features other than 'prefers-color-scheme'. If this is the only use case, with most benefit to your scenario, consider exposing a hint of such client capability, i.e. "does this device support auto detection of light or dark mode"

We understand the feature is already shipping in Chrome, but we can't endorse adding this as a general mechanism to the Web platform.

Thank you again for bringing the feature to us for review. If there is a future proposal that resolves our concerns, please don't hesitate to open another review.

[tomayac]

Likewise thanks for the review and your time!

The base problem of this proposal is the exposure of user state rather than client capabilities of user preference media features. This allows user state handling on the server side which can lead to race conditions and undesired error state and user experience. One that doesn't exist today with media features which are evaluated and handled on the client side. This is not a safe pattern and should be avoided.

In #615, you have reviewed Sec-CH-Viewport-Height, which is likewise about user state. Knowing the viewport height on the server "would enable origins to e.g., inline all the content that’s expected to appear in the viewport" (quote). This arguably causes the same "race conditions and undesired error state and user experience" if the user resizes their window at a later point after the initial load, comparable to the state change that occurs when the preferred color scheme changes.

During our call we discussed multi stakeholder interest. You mentioned of few other large Web properties that would be interested and willing to use the feature for similar benefits to those of google.com. That isn't surprising. They employ enough engineers and will be able to take the hit of implementation and support. What we don't see is evidence of other browser vendors interested to support the feature.

Well, we asked, but despite friendly pings didn't hear back:

• WebKit thread: https://lists.webkit.org/pipermail/webkit-dev/2021-May/031856.html
• Mozilla Standards Position: User Preference Media Features Client Hints Headers mozilla/standards-positions#526

We don't see evidence or use cases of user preference media features other than 'prefers-color-scheme'. If this is the only use case, with most benefit to your scenario, consider exposing a hint of such client capability, i.e. "does this device support auto detection of light or dark mode"

This reflects the popularity of the user preference media features. Up next on our list would probably be prefers-reduced-motion (i.e., Sec-CH-Prefers-Reduced-Motion). This is motivated by observations like the one below, well noting that Apple achieve this with the CSS (@media (prefers-reduced-motion)) and JS (matchMedia('(prefers-reduced-motion)')) technology available today, but envisioning further possible optimizations by tailoring the JS and CSS on the server-side, which does not even have to mean inlining (which Apple don't do), but leaving away unrelevant things in the initially served bundles, which are currently identical in both cases, but which have a different code coverage (especially main.built.js)

Browsing https://www.apple.com/apple-watch-series-7/ by scrolling from top to bottom:

• With prefers-reduced-motion set to reduce: 18.7 MB.
  
• With prefers-reduced-motion set to no-preference: 45.5 MB.
  

All that being said, we respect your resolution and thank you again for your time!