[css-color-4][css-color-adjust-1] Shielding system colors to avoid fingerprinting?

[fantasai]

The main privacy-related leak from css-color-4/css-color-adjust-1 is the way that the used value of the system colors are exposed. One way to mitigate this would be to define that the system colors always return a fixed set of values via CSSOM APIs, and that forcing colors does not affect getComptuedStyle(). That way the real used color is not Web-exposed. Do we want to do that?

The upside is that we shield the system colors from the Web: all browsers return the same values via getComputedStyle() all the time. The downside is that if the author was trying to do any interesting calculations to choose additional colors based on the user's preferred color palette, we're giving them bogus answers. (Also it's probably a bit more complicated to implement.)

[heycam]

One way to mitigate this would be to define that the system colors always return a fixed set of values via CSSOM APIs, and that forcing colors does not affect getComptuedStyle(). That way the real used color is not Web-exposed. Do we want to do that?

We'd need to use these fixed values in canvas, too, at least.

What's the current compat status of the system colors? Would it be safe to use a fixed set of colors unconditionally, or perhaps dependent only on dark mode / high contrast mode, which can be detected with the MQs? (I seem to remember that at one point the system colors were deprecated, and then they were undeprecated.)

[svgeesus]

(I seem to remember that at one point the system colors were deprecated, and then they were undeprecated.)

Right. They were all deprecated in CSS Color 3, then some of them un-deprecated in CSS Color 4

[css-meeting-bot]

The CSS Working Group just discussed [css-color-4][css-color-adjust-1] Shielding system colors to avoid fingerprinting.

Official minutes

The full IRC log of that discussion

<dael> Topic: [css-color-4][css-color-adjust-1] Shielding system colors to avoid fingerprinting
<dael> github: https://github.com//issues/5710
<dael> fantasai: We were going over the privacy considerations for color adjust
<dael> fantasai: main issue was way system colors were exposed through systemColor keywords or when we force b/c gCS returns used value rather than computed
<leaverou> q+
<dael> fantasai: Three options. 1) can't fix. 2) computed value gets returned not the used. Not sure webcompat implications. 3) return a value to gCS which is not nec the value actually used
<chris> q+
<Rossen_> q?
<leaverou> q-
<dael> fantasai: Could return the same standard values. For forced colors return the color that would have been the color before we forced
<astearns> s/CSS Contain/CSS Scoping/
<dael> fantasai: Question is do we want to do something like that? Upside is reduce fingerprinting. Down is if author wants to do something based on system colors they can't do calculation based on that
<dael> chris: I fairly recently added something for deprecated system colors which was mandatory mapping to non-deprecated. I'm not sure if that helps but it would mean used color would be same as a non-derecated colors
<Rossen_> q?
<dael> fantasai: This is about non-deprecated. It reduces number of keywords but problem still exists
<fantasai> s/It/That/
<emilio> ack chris
<leaverou> if you're interested in web compat, system colors are only used in 0.2% of pages (any system color)
<dael> Rossen_: With options 2 & 3 I'm curious if system colors will be that useful at that point. Lot of guidance on how to use system colors, esp in settings of forced colors where people encouraged to use system colors elsewhere if they take colors in their own hands
<emilio> q+
<dael> Rossen_: With that in mind I'm not sure how attainable 2 and 3 will be from author capability
<dael> fantasai: Can still use system color, just can't use a variation on the system color. You can't compute a value based on actual value
<dael> Rossen_: That was my point
<dael> chris: I think that's reasonable. I don't see call to modify system colors. Point is they're semi-standard. If you change no benefit over spec directly
<dael> leaverou: I checked web compat and system colors are only in 0.2% of pages in http archive
<dael> Rossen_: That's quite a bit
<Rossen_> ack emilio
<dael> emilio: I was going to point out opposite. Probably we cannot change. We need to return an actual color. Even though not used explicitly they're used by default in UA. Pretty sure people look at gCS on a select and expect a color back
<smfr> q+
<iank_> q+
<leaverou> There is also data per color here: https://docs.google.com/spreadsheets/d/1sMWXWjMujqfAREYxNbG_t1fOJKYCA6ASLwtz4pBQVTw/edit#gid=279222916
<dael> Rossen_: Related, if we restrict system colors to keywords only would using them as part of a color function still function?
<dael> TabAtkins: Yeah. Exposing actual color to scripts is the thing. Using them is fine
<Rossen_> ack smfr
<chris> system colors only used for links now, https://drafts.csswg.org/css-color-4/#sample
<dael> smfr: If you ahev special behavior for system colors you have to track through color modifications like lighter and darker. You'd have to mask it all the way through
<dael> TabAtkins: Yep. Similar to :visited which is tricky
<dael> chris: We changed user stylesheet for light/dark. Only three uses remain ^
<Rossen_> ack iank_
<chris> q?
<dael> iank_: I wanted to say I'm a little concerned about compat situation of changing gCS. As a webdev I did see code which would explicitly read out the color from gCS to determine if a11y mode is on. Some investigation would have to check if major web properties broke under a11y modes with this change
<dael> TabAtkins: Ultimately if we say no it's okay. It just needs to be doc as a fingerprinting risk. If we're okay with it we can leave as is. It's the only fingerprinting surface in the spec
<dael> iank_: Not saying no, saying compat risk
<dael> florian: Given these are maintained they could switch. If they are not there's something you need
<florian> s/something you need/a risk indeed/
<dael> Rossen_: Sounds like there's go with option 1 which is say no and document the fingerprinting or take another option based on additional research and experimentation
<dael> Rossen_: As a group are we comfortable with option 1? Objections to say no change on this and we'll document the fingerprinting exposure?
<dael> smfr: I need to talk to privacy people before I agree
<dael> Rossen_: In that case we'll leave it open as is. In both cases we need more data before we resolve
<dael> Rossen_: smfr can you follow up and we bring it back next week?
<dael> smfr: Sure.
<dael> Rossen_: Let's do that. We'll give the Apple folks a week and depending on how things go we may pick option 1

[svgeesus]

<dael> Rossen_: As a group are we comfortable with option 1? Objections to say no change on this and we'll document the fingerprinting exposure?
<dael> smfr: I need to talk to privacy people before I agree
<dael> Rossen_: In that case we'll leave it open as is. In both cases we need more data before we resolve
<dael> Rossen_: smfr can you follow up and we bring it back next week?
<dael> smfr: Sure.
<dael> Rossen_: Let's do that. We'll give the Apple folks a week and depending on how things go we may pick option 1

I know there was the seasonal festivities break but it has been more than a week so yes, lets get this resolved. The fingerprinting exposure is already mentioned in Security and Privacy Considerations. I'm therefore fine with option 1.

[sysrqb]

I'm providing a privacy review of this draft.

The information leakage in the forced-color property is quite concerning. As the feature is specified, the privacy and security risk is not only fingerprinting, but it is potentially revealing information about a user's health/physical condition. I understand the significant benefits in providing a mechanism for adjusting sites to meet the needs of the user, however it seems the current design allows a web page to abuse this information, too. I recommend considering options (2) or (3). Alternatively, the "correctness" of the response from getComputedStyle seems like functionality that could be gated behind a permission prompt.

Separately, regarding the color-scheme, I recommend limiting the addition of new schemes in the future.

[npdoty]

+1 that this is an information disclosure risk, not just a fingerprinting risk, especially if (as the IRC log above suggests), there are known direct cases of sites using getComputedStyle to detect whether the user has an accessibility mode turned on.

I would think that for color (as opposed to other properties that affect layout), lying in getComputedStyle is an effective mitigation, as in :visited/history sniffing.

In this case it might also require limitations on new media queries.

[css-meeting-bot]

The CSS Working Group just discussed [css-color-4][css-color-adjust-1] Shielding system colors to avoid fingerprinting?.

The full IRC log of that discussion

<dael> Topic: [css-color-4][css-color-adjust-1] Shielding system colors to avoid fingerprinting?
<dael> github: https://github.com//issues/5710
<dael> chris: Looks like nobody is in favor of close as is
<dael> chris: Other options are lie in the OM. If you've got another color scheme OM will say it's the standard. Preserves privacy but means you can't do anything clever if someone has a pallate and you want to do some coloring. If you need adaptations those won't work
<dael> florian: Was there another option?
<dael> chris: There were 3 but I've forgotten the middle one.
<dael> chris: Can someone read fantasai's second option?
<dael> TabAtkins: 1 is don't fix, 2 is we return computed value not used value. 3 is just lie
<dael> chris: Keyword seems fine. I don't know how compat that is with deployed content
<dael> TabAtkins: Problem is canvas bg color is a system color across browsers. Text too. Behavior change in any default colored text. Without fixing that this is a no go. Most text would show as canvas not blakc
<dael> chris: Yes
<dael> chris: What is impact of lie in OM?
<dael> TabAtkins: Existing visited tags are bad. I don't know if they get more bad.
<dael> chris: We have something that says all others deprecated get you an undeprecated one. That's a type of lying
<dael> fantasai: Sort of but not liying about color used. Difference between them is in general links are viisted or unvisited. When we're lying we're saying it's unvisited color. That doesn't end in author trying to calculate a contrasting color
<dael> fantasai: Generally slight change in color. If author calc against visited color they won't come up with one that won't work. Here if page syas I'm black text on white background and system comes up with white text, balck BG you get an unreadable page
<fremy> ^ I strongly agree with fantasai here
<dael> chris: Given everything comes doesn to canvas bg and text those are supposed to react to dark/light mode changes.
<dael> fantasai: It might not be exactly those colors
<dael> fantasai: And while light/dark mode will tell you canvas text vs canvas it won't tell you button colors. You won't know if they're inverted.
<dael> TabAtkins: I think fantasai makes a great point. Lying is more technically difficult and when it comes up will be worse UX. I suspect we want to do something fancy for root and otherwise go with returning system color. That will let people know there is a system color and they can use color modifications instead of calc directly
<smfr> q+
<dael> TabAtkins: Worse case code falls over b/c gets a string instead of rgb. That's better than black text balck bg
<dael> fremy: Then you have to keep track. If you have a color function. you have to keep track of it. You'd need to pass flag everywhere
<dael> smfr: Can't you detect color by using it as fill in canvas and reading back? Or do you have to take canvas as well
<dael> leaverou: Sounds messy
<smfr> s/take canvas/taint the canvas/
<dael> TabAtkins: We lie throughout the thing and render colors worthless for almost everything or we close no change. Doesn't sound like anything in middle is a useful result
<dael> chris: Some objections to close no change from privacy but they weren't aware of ramifications of others
<dael> TabAtkins: If they think there's something smart on the table this is worth solving. But it's not solvable w/o breaking the feature
<dael> chris: TabAtkins you mentioned somethign smart with canvas text and bg? What was that?
<dael> TabAtkins: Tracking and returning sytem color keywords which is complicated on canvas. If it was from root canavs or canvas text return black or white. Even that's not correct. Still exposes some colors. I could see a slight fancy way but it's patching over issues and the attempted solution is bad
<fantasai> It was a web-copat mitigation for returning keywords from gCS
<dael> astearns: Sounds to me like we're down to lying in gCS is the way to avoid privacy risks, but we don't want that.
<dael> TabAtkins: And by don't want, it would give users bad results and be very complicated to impl
<dael> astearns: What if chris or TabAtkins put a comment in this issue outlining the problems we see with lying in gCS and as for privacy to re-review as to if the drawbacks to the solution are worth mitigation
<dael> chris: Sounds good
<dael> fantasai: Also worth talking to a11y folks b/c they're the ones effected
<dael> astearns: Outline options and drawbacks, submit for re-review

See formatted official minutes.

[fantasai]

This is a summary of the state of the CSSWG discussion:

The CSS system colors, in forced colors mode, are required to match the user-required colors so that the page can style itself in that color scheme. They also are required to change based on color-scheme, so that the default text and background colors match the user preference.

The used values of these colors are exposed via getComputedStyle(), which creates a privacy leak.

Note that the following information is already provided explicitly via Media Queries:

• Whether forced colors mode is on
• What color scheme the reader prefers

Thus the additional privacy leak presented by getComputedStyle() is not of a new type of information, but rather merely a greater amount of information for users that are already segmentable via MQs.

There are three ways forward here:

1. Continue to return the used values in getComputedStyle().
2. Return the system color keywords rather than the colors themselves.
3. Lie about the colors in getComputedStyle().

For the closely-related case of the :visited pseudo-class, we took approach (3).

• First, because it was a much more severe data leak (of the user's browsing history).
• Secondly, because the impact of lying about :visited colors by returning :link colors is not very severe. Because :link and :visited are almost always styled very similarly in terms of color contrast, even if the author calculates some colors to use based on the returned :link color rather than the true :visited color, the text is most probably still readable.

But for the case of forced colors and color schemes, this approach is much more dangerous to the readability of the page. If the page believes itself to be rendered as dark text on a white background, but the system forces it to white text on a black background without, and the author calculcates custom color variants based on its own colors rather than the true colors, the page could become unreadable. Note that while the author can guess that this has happened for the page overall by combining the prefers-color-scheme media query with the forced-colors media query, these queries will only tell the author about the contrast nature of the Canvas and CanvasText colors, and not of button colors or other elements on the page, whose contrasts could be inverted.

Meanwhile the concern about option (2) is that, because of the existence of color schemes, all of the elements on the page are now expressed using system colors in the default UA stylesheet, whereas before they were all expressed as absolute colors. Because of this history of absolute colors returned by getComputedStyle(), the Web expects getComputedStyle() to return absolute colors for any element styled in its default colors: changing it to return keywords is therefore likely a change that we cannot make without breaking many pages.

The current conclusion of the CSSWG is thus that we believe that the risk of breaking pages (either via JS errors or by unreadable colors) for users employing forced colors is greater than the harm from increased fingerprinting entropy for those users.

We submit this analysis for review by Accessibility, Privacy, and the TAG.

[annevk]

Somewhat aside: I thought :visited wasn't solved yet? See #3012.

More on topic: doesn't this leak through canvas as well? 2/3 won't work there.

I think part of the question here is how the OS allows customization of these. If these are color pickers of the <input type=color> variety, it seems highly likely users that use this kind of thing would make themselves fingerprintable.

[fantasai]

More on topic: doesn't this leak through canvas as well? 2/3 won't work there.

@annevk Yes, as smfr noted on the call implementations would have to taint the canvas, which also means fingerprinting mitigations here would be very complex to implement as well.

If these are color pickers of the variety

IIRC you can choose your own colors (and for some users this may be necessary), but there are several pre-configured themes.

[svgeesus]

Given the TAG response on this issue what is the next step? Do we need to evaluate the scale of Web-incompatible breakage if we return keywords? Or do a trial of the breakage from the "CSSOM lies about it" option?

[fantasai]

@svgeesus It looks like the TAG response lists the possible behaviors in the same preferential order as we did... but it doesn't seem like they're issuing any particular recommendation here and definitely their minutes show no discussion about the downsides we identified with each and no comparison of the options to determine the best realistic way forward. So afaict, they have no actionable feedback for us, and we should continue with the “status quo” option, given the listed (assumed probably fatal) downsides of the other options.

[svgeesus]

I agree, just wanted to be sure.

[svgeesus]

So that rationale should probably be added to the security and privacy appendix.

[tabatkins]

@svgeesus Added some language summarizing the reasoning from Elika's earlier comment, and linked back to here. That work for you?

[sysrqb]

This is a summary of the state of the CSSWG discussion:

The CSS system colors, in forced colors mode, are required to match the user-required colors so that the page can style itself in that color scheme. They also are required to change based on color-scheme, so that the default text and background colors match the user preference.

The used values of these colors are exposed via getComputedStyle(), which creates a privacy leak.

Note that the following information is already provided explicitly via Media Queries:

* Whether forced colors mode is on

* What color scheme the reader prefers

Thus the additional privacy leak presented by getComputedStyle() is not of a new type of information, but rather merely a greater amount of information for users that are already segmentable via MQs.

Yes, I agree, but we're looking for solutions that limit harm (not necessarily eliminate it). However, I do agree with you now that lying is not a good solution and my original concern was misplaced. Thank you for your detailed response, it was very helpful.

Specifically, just so my thoughts on this are clear: as mentioned above, in the three proposed options the state of forced-colors would not be modified, therefore the privacy leak is inherent in this accessibility feature and it has nothing to do with the resulting styling (via gCS). The modified styling when forced-colors is enabled is likely UA/OS specific, but that information is already available to JS from other sources.

In the future I would like to see a privacy-preserving MQ mechanism that doesn't leak so much information about the UA/OS, but I know that this draft should not be a critique on MQ in general, so we can move on from this.

The current conclusion of the CSSWG is thus that we believe that the risk of breaking pages (either via JS errors or by unreadable colors) for users employing forced colors is greater than the harm from increased fingerprinting entropy for those users.

I agree this is a reasonable assessment.

[svgeesus]

Added some language summarizing the reasoning from Elika's earlier comment, and linked back to here. That work for you?

Perfect