diff --git a/act-rules-format.bs b/act-rules-format.bs index 748cc1f7..19e0528d 100644 --- a/act-rules-format.bs +++ b/act-rules-format.bs @@ -15,9 +15,9 @@ Markup Shorthands: markdown yes Introduction {#intro} ===================== -There are currently many tools available which aid their users in testing web content for conformance to accessibility standards such as the Web Content Accessibility Guidelines (WCAG). As the web develops in both size and complexity, these tools are essential for managing the resources available on the web. +There are currently many tools available which aid their users in testing web content for conformance to accessibility standards such as the [Web Content Accessibility Guidelines (WCAG)](https://www.w3.org/WAI/standards-guidelines/wcag/) [[WCAG]]. As the web develops in both size and complexity, these tools are essential for managing the resources available on the web. -This format is intended to enable a consistent interpretation of how to test for accessibility requirements to avoid conflicting results of accessibility tests. It is intended to be used to describe both manual accessibility tests as well as automated tests performed by accessibility testing tools (ATTs). +This format is intended to enable a consistent interpretation of how to test for [accessibility requirements](#structure-accessibility-requirements) to avoid conflicting results of accessibility tests. It is intended to be used to describe both manual accessibility tests as well as automated tests performed by accessibility testing tools (ATTs). Describing how to test certain accessibility requirements will result in accessibility tests that are transparent with test results that are reproducible. The Accessibility Conformance Testing (ACT) Rules Format defines the requirements for these test descriptions, known as Accessibility Conformance Testing Rules (ACT Rules). @@ -25,11 +25,11 @@ Describing how to test certain accessibility requirements will result in accessi Scope {#scope} ============== -The ACT Rules Format defined in this specification is focused on the description of rules applicable to content created using web technologies, such as HTML, CSS, WAI-ARIA, SVG and more, including digital publishing. The ACT Rules Format, however, is designed to be technology agnostic, meaning that it can conceivably be used to describe test rules for other technologies. +The ACT Rules Format defined in this specification is focused on the description of rules applicable to content created using web technologies, such as [Hypertext Markup Language](https://www.w3.org/TR/html/) [[HTML]], [Cascading Style Sheets](https://www.w3.org/TR/CSS/) [[CSS2]], [Accessible Rich Internet Applications](https://www.w3.org/WAI/standards-guidelines/aria/) [[WAI-ARIA]], [Scaleable Vector Graphics](https://www.w3.org/TR/SVG/) [[SVG2]] and more, including digital publishing. The ACT Rules Format, however, is designed to be technology agnostic, meaning that it can conceivably be used to describe test rules for other technologies. -For instance, the ACT Rules Format can be used to describe ACT Rules dedicated to testing the accessibility requirements defined in WCAG, which are specifically designed for web content. +For instance, the ACT Rules Format can be used to describe ACT Rules dedicated to testing the [accessibility requirements](#structure-accessibility-requirements) defined in WCAG, which are specifically designed for web content. -Other accessibility requirements applicable to web technologies can also be testable with ACT Rules. For example, ACT Rules could be developed to test the conformance of web-based user agents to the User Agent Accessibility Guidelines. However, the ACT Rules Format would not necessarily be suitable to describe tests for the conformance of a non-web-based user agent. +Other accessibility requirements applicable to web technologies can also be testable with ACT Rules. For example, ACT Rules could be developed to test the conformance of web-based user agents to the [User Agent Accessibility Guidelines](https://www.w3.org/WAI/standards-guidelines/uaag/) [[UAAG20]]. However, the ACT Rules Format would not necessarily be suitable to describe tests for the conformance of a non-web-based user agent. ACT Rule Structure {#structure} @@ -47,7 +47,7 @@ An ACT Rule MUST be written in plain language. It MUST consist of the following * [Limitations, Assumptions or Exceptions](#structure-limitations-assumptions-exceptions), if any * [Accessibility Support](#acc-support) (optional) * [Aspects Under Test](#input-aspects) -* [Test Procedure](#test-proc) +* [Test Definition](#test-def) Unique Identifier {#unique-identifier} ---------------------------------------- @@ -64,7 +64,7 @@ An ACT Rule MUST have a description that is in plain language and provides a bri Accessibility Requirements {#structure-accessibility-requirements} ------------------------------------------------------------------- -An ACT Rule MUST identify the accessibility requirements to which the rule maps. For example, WCAG 2.0 Success Criterion 1.1.1. An ACT Rule is a complete or partial test for one or more accessibility requirements. +An ACT Rule MUST identify the accessibility requirements to which the rule maps. For example, [WCAG 2.0 Success Criterion 1.1.1](https://www.w3.org/WAI/WCAG20/quickref/#text-equiv) [[WCAG]]. An ACT Rule is a complete or partial test for one or more accessibility requirements. Outcomes from an ACT Rule MUST be consistent with the accessibility requirement, e.g. a rule only returns the outcome Fail when the content fails the accessibility requirement. This means that the rule maps to the accessibility requirement, as opposed to it merely being related to the requirement, thematically or otherwise. @@ -74,7 +74,7 @@ The actual definition of specific accessibility requirements is beyond the scope Limitations, Assumptions or Exceptions {#structure-limitations-assumptions-exceptions} ------------------------------------------------------------------- -An ACT Rule MUST list any limitations, assumptions or any exceptions for the test, the test environment, technologies being used or the subject being tested. For example, a rule that would partially test WCAG 2.0 Success Criterion 1.4.3 Contrast (Minimum) based on the inspection of CSS properties could state that it is only applicable to HTML text content stylable with CSS and does not support images of text. +An ACT Rule MUST list any limitations, assumptions or any exceptions for the test, the test environment, technologies being used or the subject being tested. For example, a rule that would partially test [WCAG 2.0 Success Criterion 1.4.3 Contrast (Minimum)](https://www.w3.org/WAI/WCAG20/quickref/#visual-audio-contrast-contrast) based on the inspection of CSS properties could state that it is only applicable to HTML text content stylable with CSS and does not support images of text. Sometimes there are multiple plausible ways that an accessibility requirement can be interpreted. For instance, it is not immediately obvious if emoji characters should be considered "text" or "non-text content" under WCAG 2.0. Whatever the interpretation is, this MUST be documented in the rule. @@ -82,9 +82,9 @@ Sometimes there are multiple plausible ways that an accessibility requirement ca Accessibility Support {#acc-support} =========================== -ACT Rules are designed to test the conformance of content using web technologies to accessibility requirements. However, not every feature of a web technology is implemented in all assistive technologies or user agents that a website may need to support. The concept of [accessibility supported](https://www.w3.org/TR/WCAG20/#accessibility-supporteddef) use of a Web technology is defined in [[WCAG20]]. Because of this, ACT Rules are not necessarily applicable in all test scenarios. For instance, a web page that has to work in assistive technologies that have no WAI-ARIA support, wouldn’t be tested with an ACT Rule that relies on WAI-ARIA support, since this could lead to false positive results. +ACT Rules are designed to test the conformance of content using web technologies to accessibility requirements. However, not every feature of a web technology is implemented in all assistive technologies or user agents that a website may need to support. The concept of [accessibility supported](https://www.w3.org/TR/WCAG20/#accessibility-supporteddef) use of a Web technology is defined in WCAG [[WCAG]]. Because of this, ACT Rules are not necessarily applicable in all test scenarios. For instance, a web page that has to work in assistive technologies that have no WAI-ARIA [[WAI-ARIA]] support, wouldn’t be tested with an ACT Rule that relies on WAI-ARIA support, since this could lead to false positive results. -Even within a rules group, certain individual rules are not always applicable. This leaves one fewer solution for passing that particular rule group. Because of this, ACT Rules have to be atomic, meaning an ACT Rule MUST NOT rely on other rules to be part of the same test scenario. +Even within a [rules group](#grouping), certain individual rules are not always applicable. This leaves one fewer solution for passing that particular rule group. Because of this, ACT Rules have to be atomic, meaning an ACT Rule MUST NOT rely on other rules to be part of the same test scenario. To support users of ACT Rules in properly defining their test scenarios, an ACT Rule SHOULD include a warning if there are significant accessibility support concerns known about a rule. @@ -92,9 +92,9 @@ To support users of ACT Rules in properly defining their test scenarios, an ACT Aspects Under Test {#input-aspects} =================================== -ACT Rules can operate on different aspects of the subject being tested. An aspect is a distinct part of the test subject itself or its underlying implementation. For example, rendering a web page to an end user involves multiple different technologies, some or all of which may be of interest to an ACT Rule. Some rules need to operate directly on the HTTP messages exchanged between a server and a client, while others need to operate on the DOM tree exposed by a web browser. Some rules may even need to operate on several aspects simultaneously such as both the HTTP messages and the DOM tree. +ACT Rules can operate on different aspects of the subject being tested. An aspect is a distinct part of the test subject itself or its underlying implementation. For example, rendering a web page to an end user involves multiple different technologies, some or all of which may be of interest to an ACT Rule. Some rules need to operate directly on the [Hypertext Transfer Protocol](https://tools.ietf.org/html/rfc7230) [[http11]] messages exchanged between a server and a client, while others need to operate on the [Document Object Model](https://dom.spec.whatwg.org) [[DOM]] tree exposed by a web browser. Some rules may even need to operate on several aspects simultaneously such as both the HTTP messages and the DOM tree. -An ACT Rule MUST include a description of all the aspects under test by the rule. Each aspect MUST be discrete with no overlap between the aspects. Some aspects are already well defined within the context of web content, such as HTTP messages, DOM tree, and CSS styling, and do not warrant a detailed description. Other aspects may not be well defined or even specific to web content. In these cases, an ACT Rule SHOULD include either a detailed description of the aspect in question or a reference to that description. +An ACT Rule MUST include a description of all the aspects under test by the rule. Each aspect MUST be discrete with no overlap between the aspects. Some aspects are already well defined within the context of web content, such as HTTP messages, DOM tree, and CSS styling [[CSS2]], and do not warrant a detailed description. Other aspects may not be well defined or even specific to web content. In these cases, an ACT Rule SHOULD include either a detailed description of the aspect in question or a reference to that description. Common Aspects {#input-aspects-common} @@ -102,70 +102,70 @@ Common Aspects {#input-aspects-common} ### HTTP Messages ### {#input-aspects-http} -The HTTP messages exchanged between a client and a server as part of requesting a web page may be of interest to ACT Rules. For example, analyzing HTTP messages to perform validation of HTTP headers or unparsed HTML and CSS. +The HTTP messages [[http11]] exchanged between a client and a server as part of requesting a web page may be of interest to ACT Rules. For example, analyzing HTTP messages to perform validation of HTTP headers or unparsed HTML [[HTML]] and CSS [[CSS2]]. ### DOM Tree ### {#input-aspects-dom} -The DOM tree constructed from parsing HTML, and optionally executing DOM manipulating JavaScript, may be of interest to ACT Rules to test the structure of web pages. In the DOM tree, information about individual elements of a web page, and their relations, becomes available. +The DOM [[DOM]] tree constructed from parsing HTML [[HTML]], and optionally executing DOM manipulating JavaScript, may be of interest to ACT Rules to test the structure of web pages. In the DOM tree, information about individual elements of a web page, and their relations, becomes available. -The means by which the DOM tree is constructed, be it by a web browser or not, is not of importance as long as the construction follows [[DOM]]. +The means by which the DOM tree is constructed, be it by a web browser or not, is not of importance as long as the construction follows the [Document Object Model](https://dom.spec.whatwg.org) [[DOM]]. ### CSS Styling ### {#input-aspects-css} -The computed CSS styling resulting from parsing CSS and applying it to the DOM may be of interest to ACT Rules that wish to test the web page as presented to the user. Through CSS styling, information about the position, the foreground and background colors, the visibility, and more, of elements becomes available. +The computed CSS [[CSS2]] styling resulting from parsing CSS and applying it to the DOM [[DOM]] may be of interest to ACT Rules that wish to test the web page as presented to the user. Through CSS styling, information about the position, the foreground and background colors, the visibility, and more, of elements becomes available. -The means by which the CSS styling is computed, be it by a web browser or not, is not of importance as long as the computation follows any applicable specifications that might exist, such as [[CSSOM]]. +The means by which the CSS styling is computed, be it by a web browser or not, is not of importance as long as the computation follows any applicable specifications that might exist, such as the [CSS Object Model](https://www.w3.org/TR/cssom/) [[CSSOM]]. ### Accessibility Tree ### {#input-aspects-accessibility} -The accessibility tree constructed from extracting information from both the DOM tree and the CSS styling may be of interest to ACT Rules. This can be used to test the web page as presented to assistive technologies such as screen readers. Through the accessibility tree, information about the semantic roles, accessible names and descriptions, and more, of elements becomes available. +The accessibility tree constructed from extracting information from both the DOM [[DOM]] tree and the CSS [[CSS2]] styling may be of interest to ACT Rules. This can be used to test the web page as presented to assistive technologies such as screen readers. Through the accessibility tree, information about the semantic roles, accessible names and descriptions, and more, of elements becomes available. -The means by which the accessibility tree is constructed, be it by a web browser or not, is not of importance as long as the construction follows any applicable specifications that might exist, such as [[CORE-AAM-1.1]]. +The means by which the accessibility tree is constructed, be it by a web browser or not, is not of importance as long as the construction follows any applicable specifications that might exist, such as the [Core Accessibility API Mappings 1.1](https://www.w3.org/TR/core-aam-1.1/) [[CORE-AAM-1.1]]. ### Language ### {#input-aspects-text} -Language, either written or spoken, contained in nodes of the DOM or accessibility trees may be of interest to ACT Rules that intend to test things like complexity or intention of the language. For example, an ACT Rule might test that paragraphs of text within the DOM tree do not exceed a certain readability score or that the text alternative of an image provides a sufficient description. +Language, either written or spoken, contained in nodes of the DOM [[DOM]] or accessibility trees may be of interest to ACT Rules that intend to test things like complexity or intention of the language. For example, an ACT Rule might test that paragraphs of text within the DOM tree do not exceed a certain readability score or that the text alternative of an image provides a sufficient description. -The means by which the language is assessed, whether by a person or a machine, is not of importance as long as the assessment meets the criteria defined in [[wcag2-tech-req#humantestable]]. +The means by which the language is assessed, whether by a person or a machine, is not of importance as long as the assessment meets the criteria defined in [[wcag2-tech-req#humantestable]] [[WCAG]]. ACT Test Definition {#test-def} =============================== -Applicability +Applicability {#test-applicability} ------------- -The applicability section is a required part of an ACT rule. It MUST contain a precise description of the parts of the test subject to which the rule applies. For example, specific nodes in the DOM tree, or tags that are incorrectly closed in an HTML document. These are known as the *test targets*. The applicability MUST only use information provided through test aspects of the same rule. No other information should be used in the applicability. +The applicability section is a required part of an ACT rule. It MUST contain a precise description of the parts of the [test subject](#output-test-subject) to which the rule applies. For example, specific nodes in the DOM [[DOM]] tree, or tags that are incorrectly closed in an HTML [[HTML]] document. These are known as the [test targets](#output-test-target). The applicability MUST only use information provided through [test aspects](#input-aspects) of the same rule. No other information should be used in the applicability. -Applicability MUST be described objectively, unambiguously and in plain language. When a formal syntax (such as a CSS selector or XPATH) can be used, that (part of the) applicability MAY use that syntax in addition to the plain language description. While testing, if nothing within the test subject matches the applicability of the rule, the result is `inapplicable`. +Applicability MUST be described objectively, unambiguously and in plain language. When a formal syntax, such as a [CSS selector](https://www.w3.org/TR/selectors-3/) [[css3-selectors]] or [XML Path Language](https://www.w3.org/TR/xpath/) [[XPATH]], can be used, that (part of the) applicability MAY use that syntax in addition to the plain language description. While testing, if nothing within the test subject matches the applicability of the rule, the result is `inapplicable`. -An objective description is one that can be resolved without uncertainty in a given technology. Examples of objective properties in HTML are element names, their computed role, the spacing between two elements, etc. Subjective properties on the other hand, are concepts like decorative, navigation mechanism and pre-recorded. Even concepts like headings and images can be misunderstood. Is this talking about the tag name, the accessibility role, it's purpose in the web page - the latter of which is almost impossible to define with objectivity. When used in applicability, these concepts MUST have an objective definition. This definition can be part of a larger glossary shared between rules. +An objective description is one that can be resolved without uncertainty in a given technology. Examples of objective properties in HTML are element names, their computed role, the spacing between two elements, etc. Subjective properties on the other hand, are concepts like decorative, navigation mechanism and pre-recorded. Even concepts like headings and images can be misunderstood. Is this talking about the tag name, the accessibility role, it's purpose in the web page? The latter of which is almost impossible to define with objectivity. When used in applicability, these concepts MUST have an objective definition. This definition can be part of a larger glossary shared between rules.
A rule for labels of HTML `input` elements may have the following expectations:
+A rule for labels of HTML `input` elements may have the following [expectations](#expectations):
Editor note - The task force is looking for feedback about whether expectations should be allowed to reference each other, or if each must be testable independently of the others. For details, see [Github issue #173](https://github.com/w3c/wcag-act/issues/173).
-When all expectations are true for a *test target* it `passed` the rule. If one or more expectations is false, the *test target* `failed` the rule. If the rule is part of a rule group, a *test target* that `failed` a rule may still meet the requirement (see Rule Grouping for details). +When all expectations are true for a test target, the test target `passed` the rule. If one or more expectations is false, the test target `failed` the rule. If the rule is part of a rule group, a test target that `failed` a rule may still meet the requirement (see [Rule Grouping](#grouping) for details).A rule for labels of HTML `input` elements may have the following expectations:
Editor note - The ACT Task Force is looking for feedback about the use of Rule groups. We are considering whether a group should be allowed to have a single rule. Though it adds some complexity, it minimizes change if rules are added in the future. Additionally, we are considering allowing a group to require more than one pass, before the group passes. As an example, this may be particularly useful for WCAG 2.0 Success Criterion 2.4.5 Multiple ways. For details, see [Github issue #161](https://github.com/w3c/wcag-act/issues/161).
-A *test target* will pass a rule group if it passes at least one of the applicable rules within that group. Likewise, a *test target* will fail the rule group, if it does not pass any of the applicable rules in the group. In the example of HTML Tables, passing the table scope rule, and failing the ARIA label and headers+id rule, would mean the *test target* passed this group of rules. +A [test target](#output-test-target) will pass a rule group if it passes at least one of the applicable rules within that group. Likewise, a test target will fail the rule group, if it does not pass any of the applicable rules in the group. In the example of HTML Tables, passing the table `scope` rule, and failing the ARIA label and `headers` + `id` rule, would mean the test target passed this group of rules. -Note that rules in a group MAY have different applicability. Because of this, not every element applicable within the group is tested by every rule. Rules MAY also be disabled during a test run due to accessibility support concerns. See [Accessibility support](#acc-support) for details. +Note that rules in a group MAY have different [applicability](#test-applicability). Because of this, not every element applicable within the group is tested by every rule. Rules MAY also be disabled during a test run due to accessibility support concerns. See [Accessibility Support](#acc-support) for details. ACT Data Format (Output Data) {#output} @@ -194,7 +194,7 @@ ACT Data Format (Output Data) {#output} With ACT Rules, it is important that data coming from different sources can be compared. By having a shared vocabulary, accessibility data that is produced by different auditors can be compared and, where necessary, aggregated. Therefore, every ACT Rule MUST express the output in a format that has all of the features described in the ACT Data Format. -Rules are tested in two steps. Firstly, the applicability is used to find a list of *Test Targets* (elements, tags or other "components") within the web page or other test subject. Then each *test target* is tested to see if all of the expectations are true. This will give the *outcome* for each *test target*. For contextual information, the output data must also include *test subject* and the *rule identifier*. +Rules are tested in two steps. Firstly, the applicability is used to find a list of [Test Targets](#output-test-target) (elements, tags or other "components") within the web page or other [test subject](#output-test-subject). Then each test target is tested to see if all of the [expectations](#expectations) are true. This will give the outcome for each test target. For contextual information, the output data must also include test subject and the [rule identifier](#unique-identifier).Editor note - The ACT Task Force is investigating to what extent a shared data format may @@ -204,13 +204,13 @@ Rules are tested in two steps. Firstly, the applicability is used to find a list This will mean that every time a rule is executed on a page, it will return a set with zero or more results, each of which MUST have at least the following properties: -- Rule Identifier (test) -- Test Subject (Web page) -- Test Target (pointer) -- Outcome (`Passed`, `Failed`, or `Inapplicable`) +- [Rule Identifier](#unique-identifier) (test) +- [Test Subject](#output-test-subject) (Web page) +- [Test Target](#output-test-target) (pointer) +- [Outcome](#output-outcome) (`Passed`, `Failed`, or `Inapplicable`)