Skip to content

Commit

Permalink
Take snapshots without Percy (#407)
Browse files Browse the repository at this point in the history
Update the visual regression snapshots to use the Storybook add-on `jest-image-snapshot` rather than Percy. We can now remove Percy from the repo entirely. The workflow now checks snapshots and commits any that have changed. Also, the play functions for stories are fixed so async actions are properly awaited. 

The loading state for the `Geolocation` component includes animation, so the snapshot doesn't match perfectly each time. To account for this, the threshold for failure for this story is increased to a 0.005% difference in the image rather than an exact match. We also use this threshold for `MapboxMap` stories because the map canvas can have slight changes.

The `MapboxMap` component takes some time to load. I wasn't able to find a good way to detect the correct time to take a snapshot, so I added a 7500ms wait for these stories to give the map time to load before taking the snapshot. Adding more robust detection logic will be part of another [item](https://yexttest.atlassian.net/browse/BACK-2843). This item will also address the snapshot for the loading story for `LocationBias`. Right now, we skip taking a snapshot for this story because the snapshot is blank. This is likely because geolocation permissions are disabled for the test runner, so the request for the location is automatically denied.

J=BACK-2832
TEST=auto
  • Loading branch information
nmanu1 committed Nov 30, 2023
1 parent 74c591f commit 182fc28
Show file tree
Hide file tree
Showing 80 changed files with 1,927 additions and 8,236 deletions.
16 changes: 0 additions & 16 deletions .github/workflows/percy-snapshots.yml

This file was deleted.

37 changes: 37 additions & 0 deletions .github/workflows/update_snapshots.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
name: Update Snapshots

on: pull_request

jobs:
update_snapshots:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
with:
ref: ${{ github.head_ref }}
token: ${{ secrets.BOT_REPO_SCOPED_TOKEN }}
- uses: actions/setup-node@v3
with:
node-version: 18.x
registry-url: 'https://registry.npmjs.org'
- run: npm ci
- run: npm run build
- run: ./tests/scripts/start-storybook.sh
env:
MAPBOX_API_KEY: ${{ secrets.MAPBOX_API_KEY }}
- run: npm run update-snapshots
- name: Push changes
id: push-changes
uses: EndBug/add-and-commit@v9
with:
message: "Update snapshots"
add: ".storybook/snapshots/__snapshots__"
push: true
pull: --rebase --autostash
default_author: github_actions
- name: Fail job if push changes
if: steps.push-changes.outputs.pushed == 'true'
uses: actions/github-script@v3
with:
script: |
core.setFailed('Snapshot updates were made. A new commit with new snapshots was pushed.')
2 changes: 1 addition & 1 deletion .storybook/main.ts
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ const config: StorybookConfig = {
const mapboxApiKey = process.env.MAPBOX_API_KEY || process.env.REACT_APP_MAPBOX_API_KEY;
return {
...config,
...(mapboxApiKey && {REACT_APP_MAPBOX_API_KEY: mapboxApiKey })
...(mapboxApiKey && { REACT_APP_MAPBOX_API_KEY: mapboxApiKey })
};
},

Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
9 changes: 9 additions & 0 deletions .storybook/snapshots/main.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
import baseConfig from '../main';
import type { StorybookConfig } from '@storybook/react-webpack5';

const config: StorybookConfig = {
...baseConfig,
stories: baseConfig.stories.map((glob) => `../${glob}`),
};

export default config;
37 changes: 37 additions & 0 deletions .storybook/snapshots/test-runner.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
import { Page } from 'playwright-core';
import { TestRunnerConfig, TestContext } from '@storybook/test-runner';
import { toMatchImageSnapshot } from 'jest-image-snapshot';

const customSnapshotsDir = `${process.cwd()}/.storybook/snapshots/__snapshots__`;

/**
* See https://storybook.js.org/docs/react/writing-tests/test-runner#test-hook-api-experimental
* to learn more about the test-runner hooks API.
*/
const renderFunctions: TestRunnerConfig = {
setup() {
expect.extend({ toMatchImageSnapshot });
},
async postVisit(page: Page, context: TestContext) {
if (context.id === 'locationbias--loading') {
return;
}

const isMapboxMapStory = context.id.startsWith('mapboxmap--');
if (isMapboxMapStory) {
await page.waitForTimeout(7500);
}

const image = await page.screenshot();
expect(image).toMatchImageSnapshot({
customSnapshotsDir,
customSnapshotIdentifier: context.id,
failureThreshold: context.id === 'geolocation--loading' || isMapboxMapStory
? 0.005
: 0,
failureThresholdType: 'percent'
});
},
};

export default renderFunctions;
9 changes: 9 additions & 0 deletions .storybook/wcag/main.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
import baseConfig from '../main';
import type { StorybookConfig } from '@storybook/react-webpack5';

const config: StorybookConfig = {
...baseConfig,
stories: baseConfig.stories.map((glob) => `../${glob}`),
};

export default config;
8 changes: 4 additions & 4 deletions .storybook/test-runner.ts → .storybook/wcag/test-runner.ts
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
import { injectAxe, checkA11y } from 'axe-playwright';
import { Page } from 'playwright-core';
import { runOnly } from './wcagConfig';
import { runOnly } from '../wcagConfig';
import { TestContext, TestRunnerConfig } from '@storybook/test-runner';

/**
* See https://storybook.js.org/docs/react/writing-tests/test-runner#test-hook-api-experimental
* to learn more about the test-runner hooks API.
*/
const renderFunctions: TestRunnerConfig = {
async preRender(page: Page) {
async preVisit(page: Page) {
await injectAxe(page);
},
async postRender(page: Page, context: TestContext) {
async postVisit(page: Page, context: TestContext) {
await checkA11y(
page,
{
Expand All @@ -37,4 +37,4 @@ const renderFunctions: TestRunnerConfig = {
},
};

export default renderFunctions;
export default renderFunctions;
2 changes: 1 addition & 1 deletion .storybook/wcagConfig.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { RunOnly } from "axe-core";
import { RunOnly } from 'axe-core';

export const runOnly: RunOnly = {
type: 'tag',
Expand Down
Loading

0 comments on commit 182fc28

Please sign in to comment.