A11y: add fixture and matcher for a11y tests with Axe

[fastfrwrd]

Implements #2457

📦 Published PR as canary version: Canary Versions

✨ Test out this PR locally via:

npm install @grafana/plugin-e2e@3.4.0-canary.2462.22402131260.0
# or 
yarn add @grafana/plugin-e2e@3.4.0-canary.2462.22402131260.0

[github-actions]

Hello! 👋 This repository uses Auto for releasing packages using PR labels.

✨ This PR can be merged and will trigger a new minor release.
NOTE: When merging a PR with the release label please avoid merging another PR. For further information see here.

[fastfrwrd]

to see how the custom matcher outputs violations, comment out this line and run the e2es (npm run playwright:test -- --grep accessibility)

[fastfrwrd]

2) [admin-wide-screen] › tests/as-admin-user/a11y/accessibility.spec.ts:4:7 › scanForA11yViolations › runs a11y audit against a basic page 

    Error: 3 Axe violations found (Threshold: 0)

    Axe Violations:
    - Rule:   Ensure every form element has a label
      ID:     label (https://dequeuniversity.com/rules/axe/4.11/label?application=playwright)
      Impact: critical
      Affected nodes:
        - <input class="" autocapitalize="none" autocomplete="off" autocorrect="off" id="react-select-2-input" spellcheck="false" tabindex="0" type="text" aria-autocomplete="list" aria-expanded="false" aria-haspopup="true" role="combobox" aria-activedescendan...="" data-testid="data-testid Dashboar..." value= ...>
    - Rule:   Ensure that lists are structured correctly
      ID:     list (https://dequeuniversity.com/rules/axe/4.11/list?application=playwright)
      Impact: serious
      Affected nodes:
        - <ul class="css-7vg4ns" aria-label="Navigation">
    - Rule:   Ensure <li> elements are used semantically
      ID:     listitem (https://dequeuniversity.com/rules/axe/4.11/listitem?application=playwright)
      Impact: serious
      Affected nodes:
        - <li class="css-njbp03">
        - <li class="css-njbp03">
        - <li class="css-njbp03">
        - <li class="css-njbp03">
        - <li class="css-njbp03">
        - <li class="css-njbp03">
        - <li class="css-njbp03">
        - <li class="css-njbp03">
        - <li class="css-njbp03">

[Copilot]

Pull request overview

Adds first-class accessibility (a11y) testing support to @grafana/plugin-e2e by integrating Axe with Playwright and providing an ergonomic assertion matcher for test authors.

Changes:

• Introduces a scanForA11yViolations fixture that runs Axe scans on the current Playwright page.
• Adds a toHaveNoA11yViolations custom matcher to produce readable assertions and output.
• Adds an E2E spec covering the fixture + matcher behavior and wires in Axe dependencies.

Reviewed changes

Copilot reviewed 6 out of 7 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
packages/plugin-e2e/tests/as-admin-user/a11y/accessibility.spec.ts	Adds E2E coverage for the new fixture/matcher (and demonstrates usage).
packages/plugin-e2e/src/types.ts	Exposes the new fixture type in the public PluginFixture API.
packages/plugin-e2e/src/matchers/toHaveNoA11yViolations.ts	Implements the custom matcher for Axe results.
packages/plugin-e2e/src/index.ts	Wires the new fixture and matcher into the exported test/expect.
packages/plugin-e2e/src/fixtures/scanForA11yViolations.ts	Implements the Axe-backed fixture.
packages/plugin-e2e/package.json	Adds Axe-related dependencies.
package-lock.json	Lockfile updates for new dependencies.

[fastfrwrd]

this set of default tags can be overridden by passing options in, which is kind of demonstrated in the test. More on how to do that here: https://github.com/dequelabs/axe-core/blob/develop/doc/API.md#options-parameter-examples

[sunker]

This looks like a great start @fastfrwrd. Added a few comments.

[ashharrison90]

lgtm 👍

let's go! 🚀

[sunker]

Just a few minor comments. I think it's fine to mark these APIs as alpha and iterate on them. We should probably avoid announcing this feature internally until it's completed though.

[sunker]

🚀

[github-actions]

Playwright test results

Image Name	Version	Result	Report
grafana-dev	13.0.0-22400427978	✅
grafana-enterprise	12.4.0	✅
grafana-enterprise	12.0.10	✅
grafana-enterprise	10.4.19	✅
grafana-enterprise	9.3.16	✅
grafana-enterprise	8.5.27	✅

Troubleshooting

404 when clicking on View report

By default, the deploy-report-pages Action deploys reports to the gh-pages branch. However, you need to take an extra step to ensure that GitHub Pages can build and serve the site from this branch. To do so:

1. Go to the Settings tab of your repository.
2. In the left-hand sidebar, click on Pages.
3. Under Source, select Deploy from a branch, then choose the gh-pages branch.

This action needs to be completed manually in order for your GitHub Pages site to be built and accessible from the gh-pages branch. Once configured, GitHub will automatically build and serve the site whenever new reports are deployed.

[grafana-plugins-platform-bot]

🚀 PR was released in @grafana/plugin-e2e@3.4.0 🚀