Standardise and improve our manual testing methodology

[dav-idc]

What

Timeline: Ongoing work, starting in November 2022
Priority level: 3 – what reduces risk of inaccessible implementations
Category: Design system fundamentals

Potential tasks

• Formalise documentation for how we perform manual accessibility testing with assistive technologies, including the accessibility acceptance criteria
• Update the accessibility testing spreadsheet to list current browsers and assistive technology
• Define a best-practice level of manual accessibility testing within the team, and decide when it must be applied
• Bolster team use of the accessibility acceptance criteria and accessibility testing spreadsheet through additional training and defining the points in the contribution process when testing should happen
• Encourage and measure use of the accessibility testing spreadsheet

Epic lead

Not yet assigned

Why

There are limitations to our processes for manual testing at the moment, many of which might be addressed by the potential tasks listed above. The main reason 'why' is consistency.

Who needs to work on this

Whole team

Who needs to review this

Whole team

Setup tasks

• Thing to do

[dav-idc]

Over the past few months, several pieces of work have gone into improving and standardising our manual accessibility testing methods.

While there's still a lot of work to do on improving our manual testing, this is a great first step!

A large portion of the work has gone into updating our manual accessibility testing template, a spreadsheet which we use to record results from manual accessibility tests. The current version of the spreadsheet template is available in Google Sheets, and is publicly viewable.

For a recent example of how this improvement work is benefiting our everyday processes, we are currently testing the upcoming Exit this Page component using the new spreadsheet.
The ongoing work can be tracked on the Exit this Page manual accessibility testing GitHub issue.

[mgifford]

Will Apple's VoiceControl be considered for testing in the future? I'm curious how this list of software (and the hardware it depends on) is selected and updated.

[dav-idc]

Hey @mgifford! At the moment macOS Voice Control considered on an ad-hoc basis, depending on the type of component we're working on and how it might interact with voice command tools. If a new component or an update may have implications for voice command tools, then we'll often focus any additional manual testing towards those tools.

Since we follow the GOV.UK Service manual guidance on testing with assistive technologies, Dragon is the default voice command tool we test with at a minimum.

However, the selection of Dragon as the minimum voice command software with which to test was based on the 2016 GOV.UK assistive technology survey, which is now several years old. Not only that, but macOS Voice Control wasn't yet released back in 2016 (I think it was released in 2019). We're hoping for an updated assistive technology survey at some point, but so far the 2016 survey is still our most recent data source.