action-visual-snapshot

This is a GitHub action that compares 2-3 sets of images and produces a diff using odiff. The idea is to have your tests generate screenshots, save those screenshots as an artifact in GitHub, compare them to a "base" (e.g. main branch) set of images and produce a diff image of the changed pixels, and then upload those diffs as an artifact as well.

Usage

There are two steps required in your GitHub workflow. First, upload your snapshots as GitHub Action artifact. path needs to reference the path where your snapshots are saved and name is the name of the artifact for GitHub.

  - name: Save snapshots
    if: always()
    uses: actions/upload-artifact@v2
    with:
      save-only: true
      path: .artifacts/visual-snapshots

Then you'll need to run the action to compare the snapshots (gcs-bucket and gcp-service-account-key are optional and will upload your images to the bucket if specified):

    visual-diff:
      if: ${{ github.ref != 'refs/heads/master' }}
      needs: acceptance
      runs-on: ubuntu-16.04

      steps:
        - name: Diff snapshots
          id: visual-snapshots-diff
          uses: getsentry/action-visual-snapshot@v1
          with:
            gcs-bucket: 'sentry-visual-snapshots'
            gcp-service-account-key: ${{ secrets.SNAPSHOT_GOOGLE_SERVICE_ACCOUNT_KEY }}

Multiple diffs

You can also have multiple sets of diff actions with some configuration inputs

  - name: Diff snapshots
    id: visual-snapshots-diff
    uses: getsentry/action-visual-snapshot@v1
    with:
      snapshot-path: .artifacts/visual-snapshots-jest
      action-name: Jest Snapshots
      artifact-name: visual-snapshots-jest

Contributing

WIP

Updating Image Gallery Template

The main template is src/template/index.ejs -- this gets "built" into a module (that is the actual template) that the action can use and inject data into.

yarn dev:gallery && open example_gallery/index.html

Files changes to src/template/index.ejs will rebuild the example_gallery/index.html file (although you currently need to save the file twice).

Publishing

Actions are run from GitHub repos so we will checkin the packed dist folder, this is done by GitHub Actions for all branches.

To do this manually, run ncc and push the results:

$ yarn dist
$ git add dist
$ git commit -a -m "prod dependencies"

Local Testing

You can run this action locally with a script similar to the following:

#!/bin/bash
set -e

rm -rf /tmp/visual-snapshots-*
rm -rf .tmp/snaps
rm -rf "<< Results Path >>"

yarn dist

GITHUB_REPOSITORY="getsentry/sentry" \
GITHUB_HEAD_SHA="<< Commit >>" \
GITHUB_HEAD_REF="getsentry/sentry/master" \
GITHUB_WORKFLOW="visual-snapshots" \
GITHUB_WORKSPACE="." \
ACTION_SAVE_ONLY="false" \
ACTION_GITHUB_TOKEN="<< GitHub Token >>" \
ACTION_ARTIFACT_NAME="visual-snapshots" \
ACTION_BASE_ARTIFACT_NAME="visual-snapshots-base" \
ACTION_THRESHOLD="0.05" \
ACTION_SNAPSHOT_PATH="<< Snapshot Path >>" \
ACTION_LOCAL_RUN="true" \
ACTION_RESULTS_PATH="<< Results Path >>" \
node ./dist/index.js