Contrast understanding docs (1.4.3, 1.4.6, 1.4.11) need guidance on when to use pixel-picking

[dbjorge]

Background and problem statement

This is a followup to discussion from #3560 and #3671, especially comments [1] [2] [3], where there was contention about what the "right" contrast testing methodology is for us to recommend in which scenarios (in understanding docs and in ACT rules). In general, there are two categories of approach:

1. "as-specified" approaches, where testers identify colors to compare using the logical color specified by the author. Examples include "reading the CSS the author wrote" or "querying user agent for the logical color of the component in question, eg, using the colors reported by window.getComputedStyle".
2. "as-rendered" (aka "pixel-picking") approaches, where testers identify colors to compare by analyzing the pixels rendered by the user agent. Examples include "manually using a color picker tool to pick representative foreground and background pixels" or "using a testing tool which analyzes a screenshot of the text/component".

"As-rendered" approach weaknesses

Generally, "as-rendered" approaches are much more problematic for inter-tester reliability than "as-specified" approaches; there is a lot of variation between user agents, operating systems, zoom levels, monitor resolutions, etc, and the variations are ever-evolving. For example, as I write this, here is an article published yesterday explaining that Microsoft just committed updates to Chrome that will make some fundamental changes to text rendering in Chrome 124+ on Windows. Pixel-picking a "representative" color also tends to involve subjectivity more often than as-specified approaches.

"As-rendered" approaches are especially problematic in cases where rendering technology integrates with displays more granularly than "per-pixel". Note 2 on the contrast ratio definition mentions one specific case of this (anti-aliasing), but this effect is even more pronounced with, for example, Subpixel rendering (@JAWS-test provided some great examples of this interaction in these PR comments)

"As-specified" approach weaknesses

"As-specified" approaches have some problems of their own.

The most contentious issue that it's realistic for authors to make reasonable-looking font choices that result in text which is perceptually thinner than 1px, where the text as a user would perceive it truly doesn't reach the author-specified color. @WilcoFiers included some examples of issues with thin fonts in this comment, using a reasonably common web font (Roboto Thin) at a reasonable text size (16px). Unfortunately, I don't think there's any usable standard means of detecting "thin fonts" like this; even for cases where the same font-weight is specified, there's a lot of variation between font definitions. There was disagreement in #3560 about whether the desire to detect this case as a violation (which would require pixel-picking) outweighs the desire for inter-tester reliability and to be consistent with existing contrast ratio note 2.

Less contentious but still problematic is that there are cases where it is technically infeasible to actually identify individual as-specified colors when evaluating a page. Some examples where this comes up in practice (not an exhaustive list):

• Images of text
• Multicolored (eg, gradient) text
• Text rendered overtop an image background
• Translucent text over an image background
• Translucent text overlapping other page elements
• Text involving complex/implementation-defined combinations of gradients (eg, text which uses combinations of box-shadow gradients to implement outlines)

Many of these cases involve scenarios where one or the other of a foreground/background color might be determinable as-specified, but not both. Many involve cases where there isn't really one single foreground color or one single background color. The guidelines/understanding docs don't really explain how to handle these today.

There are also cases that a human tester might be able to evaluate, but which are specifically infeasible for many types of automated tools to evaluate as-specified:

• Components overlapping pseudo-elements (see csswg-drafts#6689)
• Components within closed shadow DOMs
• Links with :visited styles

Proposed updates

In practice, I think we're in general agreement that we want testers to prefer using as-specified colors where possible, and to fall back to pixel-picking where it isn't possible to determine as-specified colors.

This issue tracks updating the assorted contrast understanding docs (1.4.3, 1.4.6, 1.4.11) to add informative guidance recommending which test methodology to use where, particularly:

• That testers should use an "as-specified" approach where possible (including cases where as-specified is only possible for one of "foreground color" and "background color")
  • ...with examples of at least 1 "easy case" (just check the CSS) and 1 "hard case" (eg, translucent text over a solid background, where you'd need to do a blend-mode-aware calculation)
• That testers should use an "as-rendered" approach if-and-only-if an as-specified approach is not possible for a specific part of the content being evaluated
  • ...with several examples from the list in the background section above
  • ...clarifying that for cases like @WilcoFiers 's thin text example, it would be best practice to ensure that thin text passes both as-specified and as-rendered, but that as-specified is sufficient to meet the SC requirements.
• Guidance for how to perform as-rendered testing when one or both of the foreground/background are not single colors
  • ...noting that testers should use a mix of as-specified and as-rendered where applicable (eg, for solid text color over an image background, use as-specified for the text and as-rendered for the background)
  • The specifics of this will probably require some further discussion. @mraccess77 and @JAWS-test noted what I think is a good starting point for discussion in comments [1] [2] ("extract only those foreground pixels which contrast and ask whether the text is still readable using only those"), though I think we'd need more thought/clarity on how you'd apply that in cases where both foreground and background are variable-color (eg, translucent text over an image).

The goal here would be to give a conceptual overview of testing approach, not to replace/repeat the amount of detail we'd include in, say, an ACT rule.

[mbgower]

I support this approach and the supporting information. I believe you have the following in mind, but thought I'd state what I see as the next steps explicitly:

1. Get general task force agreement with what has been written as the rationale and the proposed updates.
2. Identify if there is a best order of documents to upgrade (i.e., is it feasible and beneficial to start with the, say, 1.4.3 Understanding updates?) and get TF agreement on that.
3. Generate the first draft PR.

Even if we decide it is necessary to propose this update as a single PR, I suspect we can iterate through changes more rapidly focusing on one document at a time in the TF. However, if you're feeling is this is better tackled holistically, I'm also in support of that.

Thanks for tackling this!!

[Myndex]

Hi @dbjorge

Here is a link to general guidelines and guidance for assessing color values, including setting up and using an eye-dropper tool for sampling (not a part of WCAG).

Per that guidance, user agent calculated colors are a preferred method, followed by CSS/SVG audit, and sampling (eye dropper) being the least preferred. However, there are many cases where sampling is the only way, such as with images, screenshot evaluations, and applications or embedded systems where access to the software/source code is not available.

As such, it is still an important tool, provided it is setup and used correctly.

Note on Font Smoothing

For testing for WCAG 2, font-smoothing should not be set to "antialiased". I.e. it should be set as:

p {
  -webkit-font-smoothing: auto;
}

font-smoothing: antialiased is a faux antialias that blends to the background. Best practice is "auto" for all text smaller than 24px and lighter than weight 400 (or 500 for some fonts).