Should WAI-ARIA "spell checking rules" be semi-automated?

[annethyme]

We have a bunch of rules that are essentially "WAI-ARIA spell checking rules":

• https://auto-wcag.github.io/auto-wcag/rules/SC4-1-1+SC4-1-2-aria-allowed-attribute.html
• https://auto-wcag.github.io/auto-wcag/rules/SC4-1-2-aria-attr-valid.html
• https://auto-wcag.github.io/auto-wcag/rules/SC4-1-2-aria-state-or-property-has-valid-value.html
• https://auto-wcag.github.io/auto-wcag/rules/SC4-1-2-role-attribute-has-valid-value.html
• https://auto-wcag.github.io/auto-wcag/rules/SC4-1-2-role-has-required-states-and-properties.html
• (https://auto-wcag.github.io/auto-wcag/rules/SC4-1-2-aria-hidden-focus.html)

They check if something that looks like WAI-ARIA is valid, based on the idea that the author has indicated that there is a need for some sort of accessibility fix, but since the WAI-ARIA is not valid, the accessibility need is still unmet.

However, these are only indications that there might be an accessibility issue, which has led to discussions on whether you can actually say that failing the rule means failing a WCAG success criterion.

It seems to me that we have other rules of the same type, where they are flagging something that might indicate an issue, and letting a user review if this means there is actually a real accessibility issue:

• SC1-1-1-placeholder-text-is-valid-accessible-name #251
• SC1-1-1-filename-is-valid-accessible-name #263
• New Rule: Links with identical accessible names serve equivalent purpose #220

I think all of these fall into a category of rules that are "indications of accessibility issues".
They are not best practices, they are indications that there is a high probably that there is an actual WCAG violation.

I think we need to decide on a consistent way of treating this type of issues.
My immediate suggestion would be to make them all semi-automated: targeting the potential issues in the Applicability and asking in the Expectation whether there is another fix in place to remediate the potential accessibility issue.

Other rules that might fall into the same category:

• Rule: iframe-unique-name #225
• https://auto-wcag.github.io/auto-wcag/rules/SC3-1-2-lang-valid.html

[ShadowBB]

That is probably a better solution than simply adding an assumption that assumes that ARIA is the only way that the SC is complied to. I agree.

[dd8]

Is an ARIA conformance failure always a WCAG conformance failure?

For example, ARIA 1.1 says "Content authors MUST provide a non-empty value for required states and properties. Content authors MUST NOT use the value undefined for required states and properties, unless undefined is an explicitly-supported value of that state or property."

This is quite close to https://auto-wcag.github.io/auto-wcag/rules/SC4-1-2-role-has-required-states-and-properties.html

The code inside Firefox/WebKit that maintains the accessibility tree assumes a lot about how ARIA is used, and authors are relying on undefined behaviour if they don't follow the spec. For example, here's what happens when you have nested DIVs marked with role=heading:
https://www.powermapper.com/tests/screen-readers/headings/aria-role-heading-nested/

Even if non-conforming ARIA works in one screen reader / browser combo there are a no guarantees that it will work in a future version of the screen reader / browser, or in a different screen reader / browser combo.

The ARIA spec also describes the behaviour of conformance checkers - but that may be out of scope for auto-wcag:
https://www.w3.org/TR/wai-aria-1.1/#conformance_checkers

[annethyme]

I previously suggested making all of these "indications of accessibility issues" rules semi-automated.

After lots of discussions with different people, I have a second suggestion for a rule of thumb for these types of rules: splitting them up in two different types:

• Rules concerning features that are normally accessibility specific, e.g. WAI-ARIA: Keep them as they are, easily automatable, with an expectation saying that the feature has been used correctly.
• Rules concerning features that are not exclusively used for accessibility, e.g. alt texts, link texts, iframe titles etc. Have them in a format where the applicability is finding the elements with potential issues, and the expectation is verifying whether it actually represents and accessibility issue. These would in general be semi-automatable, or require some kind of pattern-recognition, machine-learning or the like.

@ShadowBB, how does this match what you saw in the validation process of the published rules?

[ShadowBB]

@annethyme Sorry i missed this question.

The first camp would still need the "aria is being used with the intent to comply to WCAG" assumption of course. And tools that implemented this would probably need some warning that this could cause false positives in my humble opinion. But apart from that your suggestion would solve a lot of conflicts that have arisen from the validation process.

[jeeyyy]

Closing this as it is addressed from various PR's open towards authoring assumptions for spell checking rules.