Replace selector/steps with applicability/expecations #155

WilcoFiers · 2018-01-08T10:24:10Z

Closes #154 and closes #140

EmmaJP · 2018-01-11T16:26:30Z

act-rules-format.bs


-If either one of these passed, the rule has passed. Only if both fail, does that element fail the rule.
+1. The image has an ARIA label, giving it an accessible name
+2. The accessible name is descriptive of the image
 </div>



These two examples make it very difficult to compare, as they are not equivalent.

annethyme · 2018-01-18T14:04:43Z

act-rules-format.bs

- **CannotTell**: The result of the test is unknown. Human intervention or further testing is needed.
+- **Passed**: All *expectations* for the Test Target were true
+- **Failed**: One or more *expectations* for the Test Target was false
+- **CannotTell**: The *applicability* or some of the *expectations* were inconclusive. Human intervention or further testing is needed.


"Human intervention or further testing is needed" - This implies that the default mode for all tests are automated testing.

I'd argue that we should remove CannotTell entirely.

Perhaps CannotTell could be thought of as an interim response from automated testing only. It is needed for that purpose, even though it may not be a final outcome like the other three.
My 2p.

Dropping this particular result type for now.

nitedog · 2018-01-23T10:42:15Z

act-rules-format.bs


+**available**: The isn't hidden with `display:none`, `visibility:hidden` or though `aria-hidden="true"`.


"The isn't hidden"? Maybe this should read "The label isn't hidden"?

kasperisager · 2018-01-24T11:39:27Z

act-rules-format.bs


-Selector syntax depends on the test subject type. When a formal syntax can be used, that (part of the) selector should use that syntax. When a formal selector syntax can not be used, that (part of the) selector MAY be described unambiguously in plain language.
+Applicability MUST be described objectively, unambiguously and in plain language. When a formal syntax (such as a CSS selector or XPATH) can be used, that (part of the) applicability MAY use that syntax in addition to the plain language description. While testing, if nothing on in the test subject matches the applicability of the rule, the result is `inapplicable`.


"if nothing on in"; should the "on" be removed and "in" changed to "within"?

Editorial: Grammatical error in the first sentence. I suggest, "The applicability section is a required part of an ACT rule. It MUST contain a precise description of the parts of the test subject to which the rule applies."

kasperisager · 2018-01-24T11:52:14Z

act-rules-format.bs

@@ -161,9 +161,9 @@ Each *available* `input` element, except elements with `type="hidden"`.
 Expectations
 ------------

-Each rule MUST contain one or more expectations. An expectation is a statement that must be true about each test target (see Applicability). Each expectation MUST be written in plain language. Unlike in applicability, a certain level of ambiguity is allowed in expectations.
+Each rule MUST contain one or more expectations. An expectation is a statement that must be true about each *test target* (see Applicability). Each expectation MUST be unambiguous and be written in plain language. Unlike in applicability, a certain level of subjectively is allowed in expectations. Meaning that the expectation has only one possible meaning, but that meaning isn't fully quantifiable.


cpandhi · 2018-01-24T16:26:55Z

To explain what we mean by expectations there was a discussion to add some clarification -
Each Expectation is a statement of conformance, must be discrete and testable on its own.
For example, a rule that would partially test WCAG 2.0 Success Criterion 1.4.3 (minimum contrast) could state that it only applies to HTML text content stylable with CSS and does not support images of text.

annethyme · 2018-01-25T12:50:13Z

act-rules-format.bs


-This can be used together, such as in the following example which uses CSS selector syntax to locate elements, combined with a plain language description to further filter the nodes.
+An objective description is one that can be resolved without uncertainty in a given technology. Examples of objective properties in HTML are element names, their computed role, the spacing between two elements, etc. Subjective properties on the other hand, are concepts like decorative, navigation mechanism and pre-recorded. Even concepts like headings and images can be misunderstood. Is this talking about the tag name, the accessibility role, it's purpose in the web page? The later of which is almost impossible to define with objectively. When used in applicability, these concepts MUST have an objective definition. This definition can be part of a larger glossary shared between rules.


"with objectively" --> "with objectivity"

I'm a little confused about Subjective properties. Are we saying that they must be defined objectively by the rule, for example, a corporate logo is decorative?

Yes, "corporate logo is decorative" is subjective and so it can't be in applicability. It can be an expectation.

annethyme · 2018-01-25T12:52:34Z

act-rules-format.bs

-* Test execution which clearly outlines the actions necessary to reach the result.
-* Test result indicating whether test execution step passed or failed.
-  * Description of the error causing any failing result.
+Each rule MUST contain one or more expectations. An expectation is a statement that must be true about each *test target* (see Applicability). Each expectation MUST be unambiguous and be written in plain language. Unlike in applicability, a certain level of subjectively is allowed in expectations. Meaning that the expectation has only one possible meaning, but that meaning isn't fully quantifiable.


"subjectively" --> "subjectivity"

We had a discussion to add that each Expectation is a Conformance statement and should be testable on its own. This will clarify what we mean by Expectations.

annethyme · 2018-01-25T12:57:43Z

act-rules-format.bs

@@ -185,27 +179,27 @@ Rule Grouping {#grouping}

 In accessibility testing, there are often multiple ways to solve the same problem. For instance, in HTML tables, header cells can be indicated through the `scope` attribute, by using the `headers` attributes, or by using ARIA labels. All of these separate techniques could be described in one big rule. But this creates a large, and often difficult to maintain rule. To ensure rules are kept small and atomic, they SHOULD be put into a rule-group instead.

-To meet the accessibility requirement, only one rule in a rule-group has to pass. This way, for our table example, one could write three separate rules, one for scope, one for headers+id and one ARIA labels). If a table meets any of these rules, it automatically passes the group, and the failed results of the other rules can be ignored.
+To meet the accessibility requirement, only one rule in a rule group has to pass. This way, for our table example, one could write three separate rules, one for scope, one for headers+id and one ARIA labels). If a table meets any of these rules, it automatically passes the group, and the failed results of the other rules can be ignored.


There is an orphan ")" after "ARIA labels".

WilcoFiers · 2018-01-25T14:43:35Z

Add an editor note to discuss discrete expectations.

Replace selector/steps with applicability/expecations

dbec22f

EmmaJP suggested changes Jan 11, 2018

View reviewed changes

WilcoFiers added 2 commits January 16, 2018 11:55

Clear up ambiguity in applicability

b9bab93

Update test procedure examples

b6355b4

annethyme reviewed Jan 18, 2018

View reviewed changes

Clear up "subjective" vs "ambiguous"

e3b02a4

nitedog reviewed Jan 23, 2018

View reviewed changes

kasperisager reviewed Jan 24, 2018

View reviewed changes

annethyme reviewed Jan 25, 2018

View reviewed changes

update test procedure with TF feedback

ce16ca8

WilcoFiers merged commit d25d866 into master Jan 26, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Replace selector/steps with applicability/expecations #155

Replace selector/steps with applicability/expecations #155

WilcoFiers commented Jan 8, 2018

EmmaJP Jan 11, 2018

annethyme Jan 18, 2018

kasperisager Jan 24, 2018

EmmaJP Jan 24, 2018 •

edited

WilcoFiers Jan 25, 2018

nitedog Jan 23, 2018

WilcoFiers Jan 25, 2018

kasperisager Jan 24, 2018

WilcoFiers Jan 25, 2018

WilcoFiers Jan 25, 2018

moekraft Jan 25, 2018

kasperisager Jan 24, 2018

cpandhi commented Jan 24, 2018

annethyme Jan 25, 2018

moekraft Jan 25, 2018

WilcoFiers Jan 25, 2018

annethyme Jan 25, 2018

cpandhi Jan 25, 2018

annethyme Jan 25, 2018

WilcoFiers commented Jan 25, 2018


		available: The isn't hidden with `display:none`, `visibility:hidden` or though `aria-hidden="true"`.


		Selector syntax depends on the test subject type. When a formal syntax can be used, that (part of the) selector should use that syntax. When a formal selector syntax can not be used, that (part of the) selector MAY be described unambiguously in plain language.
		Applicability MUST be described objectively, unambiguously and in plain language. When a formal syntax (such as a CSS selector or XPATH) can be used, that (part of the) applicability MAY use that syntax in addition to the plain language description. While testing, if nothing on in the test subject matches the applicability of the rule, the result is `inapplicable`.


		This can be used together, such as in the following example which uses CSS selector syntax to locate elements, combined with a plain language description to further filter the nodes.
		An objective description is one that can be resolved without uncertainty in a given technology. Examples of objective properties in HTML are element names, their computed role, the spacing between two elements, etc. Subjective properties on the other hand, are concepts like decorative, navigation mechanism and pre-recorded. Even concepts like headings and images can be misunderstood. Is this talking about the tag name, the accessibility role, it's purpose in the web page? The later of which is almost impossible to define with objectively. When used in applicability, these concepts MUST have an objective definition. This definition can be part of a larger glossary shared between rules.

Replace selector/steps with applicability/expecations #155

Replace selector/steps with applicability/expecations #155

Conversation

WilcoFiers commented Jan 8, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

EmmaJP Jan 24, 2018 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cpandhi commented Jan 24, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

WilcoFiers commented Jan 25, 2018

EmmaJP Jan 24, 2018 •

edited