Evaluate whether all requirements from the WCAG 3 Requirements doc are testable

[WilcoFiers]

This came up in discussion from the WCAG 3 Requirements subgroup (2024-01-18). There are things under the requirements section that on the surface of it seem like they might be difficult to prove as part of WCAG 3's exit criteria. We'll want to go over this again to see if anything needs to be moved to the design principles instead.

[alastc]

We'll try to team up on this one, take one each in a comment.

For each item in section 4, what would we point to in order to say "this is complete".

How would we show it is done, or show how it isn't done?

If that is hard, how would we phrase it to make it clearer? If that's impossible, should we remove it?

4.6 and 4.7 already have PRs with updates, so leave those for now.

[alastc]

I'll tackle 4.4:

Guidance should be expressed in generic terms so that they may apply to more than one platform or technology. The intent of technology-neutral wording is to provide the opportunity to apply the core guidelines to current and emerging technology, even if specific technical advice doesn't yet exist.

I think this can be tested/demonstrated.

As part of the process we can (and should) include various (web) technologies when we create methods and tests (underneath outcomes).

Therefore, if we have Outcomes written in technology-neutral wording, with multiple technologies underneath, that demonstrates meeting the requirement.

We can have (in the background) a list of technologies which we check for each outcome as it progresses.

Noting there is a separate issue on the testability of the requirement, the solution to which would be:

"Guidance is expressed in technology-neutral terms so that it can be met using different platforms or technologies."

[alastc]

For 4.3 (Multiple ways to display) we've got an updated version in https://github.com/w3c/silver/pull/733/files

Design the guidelines so that they can be presented in different accessible and usable ways or formats, to address the needs of different audiences.

If we have a "TR" version, there needs to be one or more ways of displaying the information. The intent is to have a QuickRef style, or alternative that combines the guidance in another format that can be sliced by role, technology, or other criteria. Or include an API so that other can achieve the same thing.

On the wording, to be a requirement it should be:

The Guidelines can be presented in different accessible and usable ways or formats, to address the needs of different audiences.

[jspellman]

4.1 Broad disability support can be demonstrated by highlighting guidance included in WCAG3 that was not included in WCAG 2.2. We should have at least 3 outcomes from each of the following disability groups: low or limited vision, cognitive and learning disability, and other disabilities such as blind-deaf, Deaf, and motion-related disabilities.

Silver guidance includes tests and other evaluation methods. Some guidance may use true/false verification but other guidance will use other ways of measuring and/or evaluating where appropriate so that more needs of people with disabilities may be included (for example: rubrics, sliding scale, task-completion, user research with people with disabilities, and more). This approach includes particular attention to people whose needs may better be met with a broad testing strategy, such as people with low vision, limited vision, or cognitive and learning disabilities.

Wording suggestions:

4.1 Broader disability support
The guidance includes requirements that are not available in WCAG 2.x. Some guidance may use true/false verification but other guidance will use other ways of measuring and/or evaluating where appropriate so that more needs of people with disabilities may be included (for example: rubrics, sliding scale, task-completion, user research with people with disabilities, and more). This approach includes particular attention to people whose needs may better be met with a broad testing strategy, such as people with low vision, limited vision, or cognitive and learning disabilities.

[jspellman]

4.2 Flexible maintenance and extensibility.
Create a maintenance and extensibility model for guidelines that can better meet the needs of people with disabilities using emerging technologies and interactions. The process of developing the guidance includes experts in the technology.

We can show where experts were involved in subgroups developing the guidance and what guidance addresses emerging technologies and interactions, such as XR.

Wording suggestion:

The informative documentation uses a process that is easy to update and maintain.
The process of developing the guidance includes experts who review the guidelines to check they will work for emerging technologies and interactions.

(This seems to be 2 requirements in 1, one on maintenance and one on having the right expertise.)

[jspellman]

4.5 Readability:

The core guidelines are understandable by a non-technical audience. Text and presentation are usable and understandable through the use of plain language, structure, and design. They link to instruction videos, illustrations, and how-to where available. Creation of new videos and illustrations are outside the scope of this project at this time.

This can be demonstrated by testimonials from plain language experts who can point out where good plain language techniques were used. We can highlight where we added links to videos, illustrations, and the how-to advice.

Wording suggestion:

Requirements in WCAG 3 for plain language shall be applied to WCAG 3. It is desirable for the guidelines to be understandable by the widest possible audience.

I.e. we'll need to apply the plain language guidelines to WCAG3, which will mean we have to work through any exceptions needed for technology or testing terms needed.

Note there is a US plain language guidelines doc that might be a useful reference later.

We can include elsewhere in the document:

We will strive to add links to instruction videos, illustrations, and how-to will be added where appropriate. Creation of new videos and illustrations are outside the scope of this project at this time.

[jspellman]

4.8 Scope from a recent PR:

The guidelines provide guidance and a conformance model for people and organizations that produce digital assets and technology of varying size and complexity. This includes large, dynamic, and complex websites. The associated informative materials will include guidance for a diverse group of stakeholders including content creators, browsers, authoring tools, assistive technologies, and more.

Can be demonstrated as follows:

• point out where we address needs of small organizations and large organizations.
• show where we addressed needs of large, dynamic and complex websites
• show where we provided guidance for diverse stakeholders (largely the how-to sections for product managers, designers, UX, and developers)
• show where we provided guidance for assistive technology, browsers & authoring tools.

Wording suggestion:

WCAG 3 will provide guidance and a conformance model suitable for people and organizations that produce digital assets and technology of varying size and complexity. This includes large, dynamic, and complex websites. The scope of the associated informative materials will include guidance for a diverse group of stakeholders including content creators, browsers, authoring tools, assistive technologies, and more.

[alastc]

The 4.7 motivation requirement one has been updated in a PR (for approval) to:

The Guidelines motivate organizations to go beyond minimal accessibility requirements by providing a conformance model that rewards organizations which demonstrate a greater effort to improve accessibility.

I think the logical basis for this was that a scoring system would lead people to want a higher score (compared to pass/fail).

A slight wording issue is that the W3C / WCAG can't "reward" an organization. Perhaps it would be better as "a conformance model that shows which organizations demonstrate a greater effort to improve accessibility"?

I think if there is any granularity in the conformance that goes beyond pass/fail this should be met. A score would enable that, or it could be that you meet a base level and then can optionally show which high-level areas such as qualitative checks and performing usability testing you have completed.

So long as there is a mechanism beyond pass/fail, this should be met.

[alastc]

The 4.6 regulatory environment requirement has been updated in #727.

4.6 Regulatory Environment
Requirements written to facilitate adoption into law, regulation, or policy, since this has been shown to have major impact on practice. This includes:

• making the need and purpose for the requirement clear
• a clear definition of when requirements are met.

I think that is something that can be observed in the guidelines, and something we should build into the acceptance criteria for the guidelines.

[patrickhlauke]

the W3C / WCAG can't "reward" an organization

absolutely agree. We want to steer clear from making accessibility guidelines sound like some kind of "award" (which can then also lead to people wanting to game it)