A suite of accessibility tests that can be run within the Ember testing framework
JavaScript HTML CSS
Pull request Compare This branch is 8 commits behind ember-a11y:master.
Latest commit b5ff7e6 Feb 20, 2017 @trentmwillis Released v0.2.2
Permalink
Failed to load latest commit information.
addon-test-support Add a11y-audit test helper Feb 14, 2017
addon Merge pull request #35 from ksin/master Sep 16, 2016
app Allow turnAuditOff to be defined globally Feb 17, 2017
config Upgrade all dependencies Feb 14, 2017
content-for Properly report exceptions Feb 15, 2017
docs/assets implement demo and support for noise levels Aug 18, 2016
tests Properly report exceptions Feb 15, 2017
vendor Initial Commit from Ember CLI v0.2.7 May 19, 2016
.bowerrc Initial Commit from Ember CLI v0.2.7 May 19, 2016
.editorconfig
.ember-cli
.gitignore
.jshintrc Initial Commit from Ember CLI v0.2.7 May 19, 2016
.npmignore update ember cli version, fix diffs, and specify versions for ember-t… May 23, 2016
.travis.yml Move axe-core to npm dependency Feb 15, 2017
.watchmanconfig update ember cli version, fix diffs, and specify versions for ember-t… May 23, 2016
CHANGELOG.md Released v0.2.2 Feb 20, 2017
LICENSE.md
README.md
bower.json Move axe-core to npm dependency Feb 15, 2017
ember-cli-build.js implement demo and support for noise levels Aug 18, 2016
index.js
package.json
testem.js

README.md

ember-a11y-testing

Build Status NPM Version Ember Observer Score

Ember A11y Testing is a wrapper around Deque Labs' axe-core accessibility testing engine. It automatically integrates into your testing environment by running during the afterRender step in the run loop during any acceptance tests.

If you're using Ember 1.13.0 or above, it also integrates into your development workflow by running during a component's didRender phase in non-production environments. This gives you instant feedback on if your component's are accessible in any given state.

Installation

ember install ember-a11y-testing

Usage

Testing Usage

By default, Ember A11y Testing will automatically begin running during acceptance tests. It also injects the axe object globally during development so you can run tests while developing your application as well.

Note: any tests run with Ember A11y Testing will adjust the testing container to occupy the entire screen. This is to simulate the actual application environment, as browsers adjust styles at small sizes for accessibility reasons. It will reset itself at the conclusion of testing though.

Disabling/Enabling Axe During Tests

By default, the axe-core tests only run during acceptance tests. In order to enable them for other tests, simply run the following at the beginning of your testing module:

axe.ember.turnAxeOn();

On the flip side, if you want to turn tests off, simply use:

axe.ember.turnAxeOff();

Setting Axe Test Options

You can pass specific options to be used during a11yCheck by setting them on a global testOptions property:

axe.ember.testOptions = {
  runOnly: {
    type: "tag",
    values: ["wcag2a"]
  }
};

You can see the available options in the axe-core repo.

Note: the options will stay set, until set to something different.

a11yAudit Helper

Ember A11y Testing also provides a simple helper to run accessibility audits on-demand within your test suite.

For Acceptance tests, the helper is an async test helper so you can use it like this:

import a11yAudit from 'ember-a11y-testing/test-support/audit';

// ...elided for brevity

test('Some test case', function(assert) {
  visit('/');
  a11yAudit();
  andThen(() => assert.ok(true, 'no a11y errors found!'));
});

The helper can optionally accept a "context" on which to focus the audit as either a selector string or an HTML element. You can also provide a secondary parameter to specify axe-core options.

The helper is also able to be used Integration/Unit tests like so:

import a11yAudit from 'ember-a11y-testing/test-support/audit';

// ...elided for brevity

test('Some test case', function(assert) {
  this.render(hbs`{{some-component}}`);

  let axeOptions = {
    rules: {
      'button-name': {
        enabled: false
      }
    }
  };
  return a11yAudit(this.$(), axeOptions).then(() => {
    assert.ok(true, 'no a11y errors found!');
  });
});

As you can see, the usage for all types of tests is pretty much the same. The only real difference is Acceptance tests get automatic async handling.

Development Usage

Usage in development is restricted to applications using Ember 1.13 and up, as it relies on the didRender hook of a component's life-cycle (a feature only available in versions of Ember with the Glimmer rendering engine).

That said, setup for development is as simple as it is for testing: simply install the addon.

By default, Ember A11y Testing will audit a component for accessibility each time it is rendered. This ensures that the component is still accessible even after state changes, and since the checks are scoped to a component's element, it means that any state change propagated downwards is also caught.

Inspecting Violations

When a violation is detected for a component's element, Ember A11y Testing will attempt to visually indicate the violation by updating the component's element className.

The class used is determined as follows:

1) If a component's axeViolationClassNames property is set, these names will always be used. This provides a way for you to customize the styling of a violation to your own needs.

  • axeViolationClassNames can be passed in your template, with each name as an element in a space-separated string:

      {{x-button
        id="violations__empty-button"
        data-test-selector="empty-button"
        axeViolationClassNames="outline outline--red"
      }}
  • they can also be defined as an array of names within your component:

      axeViolationClassNames: ['outline', 'outline--red']

2) By forgoing custom styling, you can rely on Ember A11y Testing's own set of default styles. These are set according to a series of configurable visualNoiseLevels spanning from 1 to 3, and a special styling is applied to element's whose default appearance would require a different set of rules for visibility.

  {{x-button
    id="violations__empty-button"
    data-test-selector="empty-button"
    visualNoiseLevel=2
  }}

The stylesheet that Ember A11y Testing gets these rules from can be found here, but below are a few sample results for a violation caused by a button without discernable text.

"Noise" Level 1:

"Noise" Level 2:

"Noise" Level 3:

At the same time, a violation error message will be logged to the console with even more detailed information as to what went wrong. The following message corresponds to the same text input element above:

Special Styling for Background-replaced elements

As mentioned, some HTML elements are considered to be "replaced" elements, wherein their rendered appearance is -- to varying degrees -- determined by the browser and/or the content that the element consists of.

An img is an example of a replaced element; it lacks the ability to be styled through background definitions, as it's background is replaced with its loaded source assets.

To get around this, Ember A11y Testing tries to detect these elements and -- unless custom class names have been specified -- apply special styling independent of the component's current noise level:

Component Hooks

Since development is not a uniform experience, Ember A11y Testing provides several hooks to help stay out of the way.

Note: these are all undefined by default.

Defining a custom callback

If you feel the logging of violations is poor or you just want to see the entire results of a component's audit, you can define a custom callback. The callback receives the results of the a11yCheck audit that is scoped to the component's element. Simply set it as axeCallback on the component in question:

axeCallback(results) {
  // do stuff with results
}
Setting options for the audit

As with testing, if you need to set custom auditing options for a component, you can do so easily. Simply set a value for the axeOptions property value:

axeOptions: { /* a11yCheck options */ }
Turning the audit off

Lastly, if you really find the audits to be cramping development, you can turn them off via a simple boolean switch:

turnAuditOff: true

Environment Options

Each of the fine-grained component hooks above can instead be defined for ALL components inside of your application's config/environment.js file. Simply supply them in a componentOptions hash on the ember-a11y-testing property of ENV.

ENV['ember-a11y-testing'] = {
    componentOptions: {
      turnAuditOff: false, // Change to true to disable the audit in development
      axeCallback: defaultAxeCallback,
      axeOptions: defaultAxeOptions,
      visualNoiseLevel: 2,
      axeViolationClassNames: ['alert-box', 'alert-box--a11y']
    }
  }
};

Future Plans

Now that your components and acceptance tests can self-audit, the next step going forward is to give helpful and meaningful feedback to developers. This means easily highlighting areas with violations and giving suggestions on how to fix and improve them. Additionally, work will be done to tackle Ember-specific accessibility issues, such as identifying actions on inaccessible elements.