investigate tooling to check for a11y issues

[jgiardino]

Investigate what tools are available that can help us flag accessibility issues that may be present in a component that is being implemented for PatternFly 4. Provide any information that would be helpful in deciding between tools, if there is more than one tool that would work.

Done Criteria:

• Select and agree on tool for UXD to use to verify a11y violations
• PR with a base solution posted against https://github.com/patternfly/patternfly-next/
• Issues opened for remaining work items

Requirements:

• Tooling should be usable without needing to write code or run scripts
• Tooling should be scriptable and able to run as part of CI jobs
• Tooling should be JavaScript framework agnostic
• Tooling should be able to analyze static html pages
• Tooling should allow for report formats that are easy to understand
• Tooling should allow for analyzing pages against WCAG2.1 AA

Nice to have:

• Tooling should be able to analyze pages with dynamic content
• Tooling should allow for analyzing pages against multiple standards like Section508, WCAG2.0 AA, WCAG2.0 AAA, etc.
• Tooling is ideally testing framework agnostic, or at least compatible with leading testing frameworks
• Tooling should be able to optionally fail a build when it finds a number of violations
• Tooling can include/exclude target sections of DOM
• Tooling can target specific rules like color contrast, accessible names, etc.

[seanforyou23]

After reviewing several approaches for testing UI for accessibility, result for most widely available and documented tooling goes to : aXe: The Accessibility Engine

About aXe Core

Through my investigation, aXe became the obvious choice for us moving forward. For many reasons, but among the top were:

1. aXe browser extension is a low barrier to entry for testing against common rulesets for regulatory compliance in the a11y space - https://chrome.google.com/webstore/detail/axe/lhdoppojpmngadmnindnejefpokejbdd and
   https://chrome.google.com/webstore/detail/axe-coconut/iobddmbdndbbbfjopjdgadphaoihpojp and https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/
2. aXe CLI further supports overnight integration
3. Integrates with selenium, mocha, phantomJS, etc. for when you need fine grain control, scripting, custom reporting, etc. (e.g. attached PR)
4. Recommended in https://www.w3.org/WAI/ER/tools/ and https://reactjs.org/docs/accessibility.html
5. Allows for targeting specific standards
6. Allows for excluding certain rules
7. Allows for specifying certain sections of DOM to analyze
8. Allows for spot testing pages
9. Does not require already having an automated testing workflow set up to use
10. aXe is open source, and actively involved in the community - hosting webcasts, posting blogs and tutorials, and responding quickly to github issues

Screenshots below to illustrate what working with this tool can look like;

Using browser extensions

Using npm scripts

Using axe-react package

Using aXe cli

https://gist.github.com/seanforyou23/7a1dd51939b142fadea795cc6a9b0a88