diff --git a/docs/product/snapshots/index.mdx b/docs/product/snapshots/index.mdx new file mode 100644 index 00000000000000..913878e394e66e --- /dev/null +++ b/docs/product/snapshots/index.mdx @@ -0,0 +1,41 @@ +--- +title: Snapshots +sidebar_order: 138 +description: Snapshot testing to catch visual changes on every pull request. +beta: true +--- + + + +Snapshots helps you catch unintended visual changes before they reach users. Sentry handles image storage, diffing, CI checks, and more. + +TODO: Arcade Walkthrough (1. status check showing diff, 2. click into sentry to see diff, 3. approve) + +## How It Works + +1. **Generate snapshots** — Your CI job produces screenshots however you like: Playwright, Paparazzi, or any tool that outputs PNGs +2. **Upload to Sentry** — `sentry-cli` uploads the snapshot directory to Sentry +3. **Sentry diffs against the baseline** — Sentry compares images against corresponding base build (TODO: link to section explaining automated comparisons). With Sentry, you do not have to manage "Golden" images and everything is determined via git metadata. +4. **Results post to your PR** — Snapshot results post to your PR\*. You can configure settings(TODO: link to settings) to control status check and comment behavior. +5. **Review and approve** — View snapshot results on the Sentry UI. If the Snapshot "failed" you can approve it from either the PR or the Sentry UI. + +\*Note: Snapshots EA only supports Github + +## Prerequisites + +- **Early Adopter access** — Enable the Early Adopter toggle in your [organization settings](/organization/early-adopter-features/). +- **Auth token** — A Sentry auth token with `project:write` scope (personal token) or `org:ci` scope (org-level token). +- **sentry-cli >= 3.3.5** — The `build snapshots` command requires version 3.3.5 or later. +- **GitHub integration** — Snapshots can be used with any VCS provider, but only Github is supported for status check integration and PR comments. Install the [Sentry GitHub App](/organization/integrations/source-code-mgmt/github/) and grant it access to your repository for PR integration. + +## Supported Platforms + +Snapshots works for any platform with a frontend and most platforms have a number of tools to help you generate snapshots. Sentry has a recommended worfklow for the following platforms: + +TODO: idk if we even want this, but putting for now + +- iOS +- Android +- React + + diff --git a/docs/product/snapshots/reviewing-snapshots/index.mdx b/docs/product/snapshots/reviewing-snapshots/index.mdx new file mode 100644 index 00000000000000..94e7323523562e --- /dev/null +++ b/docs/product/snapshots/reviewing-snapshots/index.mdx @@ -0,0 +1,44 @@ +--- +title: Reviewing Snapshots +sidebar_order: 20 +description: Review snapshot diffs on your pull request and approve changes. +--- + + + +## Status Checks + +After you upload snapshots from a pull request branch, a **Snapshot Testing** GitHub Check appears on the pull request. If no snapshots changed, the check passes. If any snapshots were added, removed, or modified, the check fails and requires approval. + +IMAGE: GitHub Check showing Snapshot Testing results on a pull request + +## Comparison Viewer + +Click **Details** on the GitHub Check to open Sentry's comparison viewer. The viewer has two panels: + +- **Sidebar** — Lists all snapshots grouped by status: modified, added, removed, renamed, and unchanged. Click a snapshot to view it. +- **Main panel** — Displays the selected snapshot's diff with three viewing modes: + - **Split** — Side-by-side view of base and current branch with a diff overlay highlighting changed pixels. + - **Wipe** — Drag a slider across the image to compare base and current branch. + - **Onion** — Overlay both images with an adjustable opacity slider. + +IMAGE: Sentry Snapshot comparison viewer showing a modified image with split diff + +## Approve Changes + +There are two ways to approve snapshot changes: + +- Click the **Approve** button on the GitHub Check Run. +- Click **Approve** in the Sentry comparison viewer. + +Approval resolves the failing check. If new changes are pushed to the PR, the check resets and requires approval again. + +## Settings + +Configure snapshot behavior in **Project Settings > Mobile Builds > Snapshots**. + +| Setting | Default | Description | +| ------------------------- | :-----: | ----------------------------------------------------- | +| Status Checks Enabled | On | Post GitHub Check Runs for snapshot comparisons. | +| Fail on Added Snapshots | Off | Require approval when new snapshots are added. | +| Fail on Removed Snapshots | On | Require approval when existing snapshots are removed. | diff --git a/docs/product/snapshots/uploading-snapshots/index.mdx b/docs/product/snapshots/uploading-snapshots/index.mdx new file mode 100644 index 00000000000000..0a431f36897896 --- /dev/null +++ b/docs/product/snapshots/uploading-snapshots/index.mdx @@ -0,0 +1,150 @@ +--- +title: Uploading Snapshots +sidebar_order: 10 +description: Structure your snapshot directory and upload from CI with sentry-cli. +--- + + + +Snapshots works for any platform with a frontend. You can view specific platform examples [here](/product/snapshots#supported-platforms). This page covers the general upload structure and metadata schema. + +## Upload command + +You upload snapshots with the `sentry-cli build snapshots` command, which takes a directory of images and optional JSON metadata files. + +```bash +sentry-cli build snapshots \ + --auth-token "$SENTRY_AUTH_TOKEN" \ + --app-id \ + --project \ +``` + +You can reference our [example workflow](#upload-with-ci) and [full flag reference](#sentry-cli-build-snapshots-reference) below. + +### Upload Structure + +Each snapshot consists of two files: a `.png` image and an optional `.json` metadata file with the same base name and path. You can optionally organize snapshots into subdirectories. If using subdirectories, the filename property in the Sentry UX will include the subdirectory path. + +``` +snapshots/ +├── homepage.png +├── homepage.json +├── settings/ +│ ├── profile.png +│ ├── profile.json +│ ├── billing.png +│ └── billing.json +``` + +Filenames are the identity key for each snapshot. Keep them stable across runs so Sentry can accurately diff head against base. + +### JSON Metadata + +You can include an optional JSON file to add metadata to each image: + +```json +{ + "display_name": "Billing Page", + "group": "Settings" +} +``` + +| Field | Type | Description | +| -------------- | ------ | ---------------------------------------------------------- | +| `display_name` | string | Human-readable label shown in the comparison viewer. | +| `group` | string | Groups related snapshots in the comparison viewer sidebar. | + +TODO: IMAGE OF DISPLAY NAME AND GROUP +TODO: if diff_threshold makes it in, add + +### Supported Formats + +Snapshots supports the following image formats: + +- PNG +- JPEG + +## Upload With CI + +For a production example, Sentry's Github action is available [here](https://github.com/getsentry/sentry/blob/master/.github/workflows/frontend-snapshots.yml). Below is an example GitHub Actions workflow you can use as a starting point. Replace the placeholder values with your own snapshot generation command, app ID, and Sentry project slug. + +```yml {filename:.github/workflows/snapshots.yml} +name: Snapshots + +on: + push: + branches: [main] + pull_request: + +env: + SNAPSHOT_OUTPUT_DIR: .artifacts/snapshots + +jobs: + snapshots: + runs-on: ubuntu-24.04 + timeout-minutes: 20 + steps: + - uses: actions/checkout@v4 + with: + # Use PR head SHA, not the merge commit — avoids phantom diffs + ref: ${{ github.event.pull_request.head.sha || github.sha }} + + # ------------------------------------------------------- + # YOUR SNAPSHOT GENERATION STEP(S) HERE + # ------------------------------------------------------- + # Generate PNG + JSON pairs into $SNAPSHOT_OUTPUT_DIR. + # Examples: + # - Playwright screenshots + # - iOS/Android screenshots + # - Laravel Dusk screenshots + # - Any tool that produces PNGs + # ------------------------------------------------------- + - name: Generate snapshots + run: + + - name: Install sentry-cli + if: ${{ !cancelled() }} + run: curl -sL https://sentry.io/get-cli/ | SENTRY_CLI_VERSION=3.3.4 sh + + - name: Upload snapshots + if: ${{ !cancelled() }} + env: + SENTRY_AUTH_TOKEN: ${{ secrets.SENTRY_AUTH_TOKEN }} + run: | + ARGS=( + --auth-token "$SENTRY_AUTH_TOKEN" + build snapshots "${{ env.SNAPSHOT_OUTPUT_DIR }}" + --app-id + --project + ) + + + sentry-cli "${ARGS[@]}" +``` + +## `sentry-cli build snapshots` Reference + +```bash +sentry-cli build snapshots [OPTIONS] --app-id +``` + +| Flag | Description | +| ------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `` | Directory containing `.png` images (and optional `.json`). | +| `--app-id ` | Identifier for the app or bundle (e.g., `web-frontend`, `ios-app`). Must be consistent across uploads — Sentry uses it to match head and base snapshots. Max 255 characters. | +| `--auth-token ` | Sentry auth token with `project:write` or `org:ci` scope. Can also be set via `SENTRY_AUTH_TOKEN`. | +| `-o`, `--org ` | Sentry organization slug. Can also be set via `SENTRY_ORG`. | +| `-p`, `--project ` | Sentry project slug. Can also be set via `SENTRY_PROJECT`. | +| `--head-sha ` | 40-character commit SHA of the snapshot build. Auto-detected in CI. | +| `--base-sha ` | Base branch commit SHA for comparison (PR only). Auto-detected in CI. | +| `--vcs-provider ` | VCS provider, e.g., `github`. Auto-detected from git remote. | +| `--head-repo-name ` | Repository in `owner/repo` format. Auto-detected from git remote. | +| `--base-repo-name ` | Base repository in `owner/repo` format (PR only, for forks). Auto-detected in CI. | +| `--head-ref ` | Head branch name. Auto-detected in CI. | +| `--base-ref ` | Base branch name (PR only). Auto-detected in CI. | +| `--pr-number ` | Pull request number (PR only). Auto-detected in GitHub Actions. | +| `--diff-threshold ` | Float between 0.0 and 1.0. Sentry only reports images as changed if the percentage of changed pixels exceeds this value. For example, `0.01` ignores changes under 1%. | +| `--no-git-metadata` | Skip automatic git metadata detection. | +| `--force-git-metadata` | Force git metadata collection even outside CI. | +| `--log-level ` | Logging verbosity: `debug`, `info`, `warn`, or `error`. | +| `--url ` | Sentry instance URL. Defaults to `https://sentry.io`. |