VRT

Visual regression testing for different scenarios. Useful for testing local sites against a dev or staging site, or testing a staging site against production before or after deployment.

Remember to add the HTTP auth credentials for relevant sites: TEST_URI=https://user:pw@test.com

Set up

• An example file for ENV variables example.env - Copy and rename to .env.
• A txt file for the URLs to test config/urls_name-of-test.txt
• A json file with BackstopJS configuration like viewports, scenarios, and debugging backstop_name-of-test.json
• A js file for the test test_name-of-test.js
• A directory for custom scripts library/engine_scripts/ to run on ready. See https://github.com/garris/BackstopJS#running-custom-scripts
• A node script in package.json for each reference and test command

Testing a list of URLs

Screenshots of multiple viewports from a list of public URLs.

REF_URI=full URL for reference files
TEST_URI=full URL for test files

See tools-vrt/backstop_anon.json for default config.

Testing click and hover behaviour

Screenshots of multiple viewports of click and hover events based on selectors. Defaults to Common Design header UI elements.

REF_URI=full URL for reference files
TEST_URI=full URL for test files

See tools-vrt/backstop_ui.jsonfor default config or to add other elements. Eg.

"scenarios": [
    {
      "clickSelector": ".search__btn"
    },

If using the click or hover interactions, scripts need to be included (Refer to Backstop examples for more info). The scripts should be added to library/engine_scripts/ and referenced by filename only Eg. "onReadyScript": "onReady.js", if "engine_scripts": "library/engine_scripts", is defined in the json config.

Testing authenticated pages

Screenshots of multiple viewports of a list of URLs that require a log in step to access. See library/engine_scripts/login.js for default values.

REF_URI=full URL for reference files
REF_LOGIN_PAGE=path to login page Eg: '/user/login'
REF_USERNAME_INPUT=username input selector Eg. '#email'
REF_PASSWORD_INPUT=password input selector Eg. '#password'
REF_SUBMIT=submit button selector Eg. '.t-btn--login'
REF_USERNAME=username for reference URL
REF_PASSWORD=password for reference URL

TEST_URI=full URL for test files
TEST_LOGIN_PAGE=path to login page Eg: '/user/login'
TEST_USERNAME_INPUT=username input selector Eg. '#email'
TEST_PASSWORD_INPUT=password input selector Eg. '#password'
TEST_SUBMIT=submit button selector Eg. '.t-btn--login'
TEST_USERNAME=username for test URL
TEST_PASSWORD=password for test URL

See tools-vrt/backstop_auth.json for default config.

See https://github.com/garris/BackstopJS for config examples.

Removing images

Screenshots of multiple viewports from a list of public URLs that replaces the images with purple or cats.

REF_URI=full URL for reference files
TEST_URI=full URL for test files

See tools-vrt/backstop_img.json for default config. Swap purpleImages.js for catImages.js in onReadyScript in the backstop_img.json config.

Setup

• Clone this repo
• npm i
• Add URLs to config/urls_anon.txt, hostname does not really matter
• Edit .env file. Add the relevant variables to .env file and change their values as needed.

Run tests

Generate reference screenshots

npm run reference-anon

Generate test screenshots

npm run test-anon

Generate reference screenshots for authenticated pages

npm run reference-auth

Generate test screenshots for authenticated pages

npm run test-auth

Generate reference screenshots for UI behaviour

npm run reference-ui

Generate test screenshots for UI behaviour

npm run test-ui

Troubleshooting

If there are permission issues with the directories for the screenshots and reports, remove the folders before running the npm commands. sudo rm -R -df data/anon/

Change debug and debugWindow to true to see the tests run in a local Chromium instance.

If an error around Chrome missing eg. Error: Failed to launch the browser process! puppeteer/chrome/linux-127.0.6533.88/chrome-linux64/chrome: error while loading shared libraries: libasound.so.2: cannot open shared object file: No such file or directory

See https://learn.microsoft.com/en-us/windows/wsl/tutorials/gui-apps#install-google-chrome-for-linux and https://learn.microsoft.com/en-us/windows/wsl/tutorials/gui-apps#install-google-chrome-for-linux