Skip to content
This repository has been archived by the owner on Dec 22, 2023. It is now read-only.

alphagov/verify-visual-regression-tests

Repository files navigation

Verify - Visual Regression Testing

This repo contains configuration for BackstopJS to detect unintended visual regressions introduced by new code.

GOV.UK Verify has closed

This repository is out of date and has been archived

HEAVILY inspired by the Digital Marketplace visual regression tests. Thanks Digital Marketplace.

Local setup

You'll need the frontend and hub running locally.

You'll also need node installed locally.

Clone this repo and run: npm install

Running tests locally

Simply run:

npm run test

Or, to use the script that does some housekeeping, run:

./run-tests.sh -e local

See the script for all available options.

With the basic setup as supplied, this will run an initial test on your localhost. Everything will fail because you don't have any reference screenshots. To get some references, simply approve your test run:

npm run approve

Or create fresh reference images before running the tests with

npm run reference

If you're using the script it will automatically generate fresh reference images if you don't have any, before running the tests.

Now you can happily change your local frontend and run tests (with npm run test) to make sure you've not broken things! Everytime you do a good change, simply npm run approve to update your reference screenshots.

Running tests in staging

Simply run

./run-tests.sh -e staging

Running tests with Concourse

Concourse automatically runs the tests against staging and publishes the HTML report to the PaaS.

If the tests fail you can use the HTML report to decide if you need to fix something or if you want to approve the diff.

Approving the tests with Concourse

To approve the last tests run the approve-visual-regression-tests job. This will promote the last test images to the reference images and re-run the tests in the pipeline.

Skipping the tests with Concourse

There may be situations where the tests fail but you don't want to approve the failures, or fix whatever made them fail. This is probably quite rare - maybe you just need to get a release out or something. To do this run the skip-visual-regression-tests-without-approval job. The tests will still fail on the next run.

Updating tests

To add new test scenarios, update config.js under the scenarios key and follow the format of other entries in the list. After your updates are merged to master Concourse will run the new tests automatically.

Licence

Unless stated otherwise, the codebase is released under the MIT License. This covers both the codebase and any sample code in the documentation.

The documentation is © Crown copyright and available under the terms of the Open Government 3.0 licence.

About

Verify visual regression tests

Topics

Resources

License

Code of conduct

Security policy

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published