Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make it clear when aXe has failed, succeeded or found errors #64

Merged
merged 6 commits into from Sep 15, 2017

Conversation

@fofr
Copy link
Contributor

@fofr fofr commented Sep 15, 2017

Easiest to review with ?w=1 to ignore whitespace changes.

Our integration tests in government-frontend couldn't easily check that aXe was present and running on each page. If aXe itself had an error we swallowed the error and CI would report that as "no accessibility errors".

Refactoring to fix this unveiled one of these false positives: a problem with restoreScroll on pages with SVGs – a bug in aXe itself. Added a tweak to prevent this from erroring in the future. See: dequelabs/axe-core#525

Use body class to indicate accessibility tests have run and their state. Classes can be used in integration tests to confirm tests are running.

See: https://github.com/alphagov/government-frontend/compare/assert-axe-runs

fofr added 5 commits Sep 14, 2017
* Classes will be used in integration tests to confirm tests are running
* eg need to confirm JS has run and not thrown any errors
* Also removes a method shortcut which set `err` to a success value
* Update called methods to handle empty lists
When `err` had a value or `results` was undefined we called the
callback but we didn’t return from the function, leading to subsequent
code erroring or requiring extra logic.

* Return early when err or no results
* Simplify logic
See: dequelabs/axe-core#525

If the code encounters an SVG it essentially runs,
document.querySelector('svg').children. This doesn't always return an
HTMLCollection object which in turn causes Array.from to error.
When there’s nothing to test that’s not really an error. Instead
confirm that axe hasn’t run with a test.
Only possible following 6369068

Make it easy to see that aXe had a problem running. In CI this error
would otherwise be ignored and it may look like accessibility tests
were passing.
@fofr fofr requested review from nickcolley and vanitabarrett Sep 15, 2017
})
)
)
} else {

This comment has been minimized.

@nickcolley

nickcolley Sep 15, 2017
Contributor

We can return early above if we flip the conditional


var bodyClass = results.violations.length === 0 ? "test-a11y-success" : "test-a11y-failed"
document.body.classList.add(bodyClass);
document.body.classList.add("test-a11y-finished");

This comment has been minimized.

@nickcolley

nickcolley Sep 15, 2017
Contributor

Rather than this can we assign a global maybe?

window.__A11Y_TEST_STATUS = 'finished'

This comment has been minimized.

@nickcolley

nickcolley Sep 15, 2017
Contributor

Ah you want to use them in integration tests... If it's not easy to do the above seems fine. Might want to add 'js-' though.

This comment has been minimized.

@fofr

fofr Sep 15, 2017
Author Contributor

Updated to use js- prefixes in 61dec17

Copy link
Contributor

@nickcolley nickcolley left a comment

Good detective work 🕵️

This was tricky to find out, thanks Paul! 💯

@fofr fofr merged commit d8f2321 into master Sep 15, 2017
1 check passed
1 check passed
continuous-integration/jenkins/branch This commit looks good
Details
@fofr fofr deleted the tweak-axe-test-runner branch Sep 15, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

2 participants
You can’t perform that action at this time.