Skip to content

Fix/visual regression tests#378

Merged
tomusdrw merged 4 commits intomainfrom
fix/visual-regression-tests
Mar 30, 2026
Merged

Fix/visual regression tests#378
tomusdrw merged 4 commits intomainfrom
fix/visual-regression-tests

Conversation

@tomusdrw
Copy link
Copy Markdown
Member

@tomusdrw tomusdrw commented Mar 30, 2026

Summary by CodeRabbit

  • Tests

    • Extended visual regression snapshot coverage to include all snapshot directories
    • Enhanced screenshot comparison configuration with pixel difference tolerance and animation handling
    • Refactored test snapshot options for improved consistency
  • Chores

    • Updated Docker build configuration and tool settings for test infrastructure
    • Added test results directory to version control and linting ignore lists

tomusdrw and others added 3 commits March 30, 2026 12:27
The new split-screen and narrow-screen snapshot tests were missing Linux
reference PNGs (only macOS snapshots existed), causing all 32 tests to
fail on CI with "snapshot doesn't exist" errors.

- Generate Linux snapshot PNGs for split-screen and narrow-screen tests
- Fix CI workflow to commit snapshots for all test files, not just
  basic-snapshots
- Add global maxDiffPixelRatio (2%) and animations: disabled to handle
  PDF rendering non-determinism between runs
- Add docker-compose build target to skip unneeded development stage

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The new split-screen and narrow-screen snapshot tests were missing Linux
reference PNGs (only macOS snapshots existed), causing all 32 tests to
fail on CI with "snapshot doesn't exist" errors.

- Generate Linux snapshot PNGs for split-screen and narrow-screen tests
- Fix CI workflow to commit snapshots for all test files, not just
  basic-snapshots
- Add global maxDiffPixelRatio (2%) and animations: disabled to handle
  PDF rendering non-determinism between runs
- Add docker-compose build target to skip unneeded development stage
- Exclude test-results from Biome checks

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@netlify
Copy link
Copy Markdown

netlify bot commented Mar 30, 2026

Deploy Preview for graypaper-reader ready!

Name Link
🔨 Latest commit 022a257
🔍 Latest deploy log https://app.netlify.com/projects/graypaper-reader/deploys/69ca534230ff220008f43818
😎 Deploy Preview https://deploy-preview-378--graypaper-reader.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Mar 30, 2026

Important

Review skipped

Review was skipped due to path filters

⛔ Files ignored due to path filters (18)
  • tools/snapshot-tests/tests/basic-snapshots.spec.ts-snapshots/homepage-outline-light-mode-linux.png is excluded by !**/*.png
  • tools/snapshot-tests/tests/basic-snapshots.spec.ts-snapshots/topbar-github-expanded-dark-mode-linux.png is excluded by !**/*.png
  • tools/snapshot-tests/tests/basic-snapshots.spec.ts-snapshots/topbar-more-clicked-dark-mode-linux.png is excluded by !**/*.png
  • tools/snapshot-tests/tests/basic-snapshots.spec.ts-snapshots/topbar-more-clicked-light-mode-linux.png is excluded by !**/*.png
  • tools/snapshot-tests/tests/basic-snapshots.spec.ts-snapshots/topbar-notes-dropdown-dark-mode-linux.png is excluded by !**/*.png
  • tools/snapshot-tests/tests/basic-snapshots.spec.ts-snapshots/topbar-notes-dropdown-light-mode-linux.png is excluded by !**/*.png
  • tools/snapshot-tests/tests/basic-snapshots.spec.ts-snapshots/topbar-notes-selected-on-dark-mode-linux.png is excluded by !**/*.png
  • tools/snapshot-tests/tests/basic-snapshots.spec.ts-snapshots/topbar-notes-selected-on-light-mode-linux.png is excluded by !**/*.png
  • tools/snapshot-tests/tests/narrow-screen.spec.ts-snapshots/narrow-layout-re-collapsed-dark-mode-linux.png is excluded by !**/*.png
  • tools/snapshot-tests/tests/narrow-screen.spec.ts-snapshots/narrow-layout-re-collapsed-light-mode-linux.png is excluded by !**/*.png
  • tools/snapshot-tests/tests/split-screen.spec.ts-snapshots/split-header-options-menu-light-mode-linux.png is excluded by !**/*.png
  • tools/snapshot-tests/tests/split-screen.spec.ts-snapshots/split-sidebar-overlay-closed-light-mode-linux.png is excluded by !**/*.png
  • tools/snapshot-tests/tests/split-screen.spec.ts-snapshots/split-sidebar-overlay-light-mode-linux.png is excluded by !**/*.png
  • tools/snapshot-tests/tests/split-screen.spec.ts-snapshots/split-view-closed-dark-mode-linux.png is excluded by !**/*.png
  • tools/snapshot-tests/tests/split-screen.spec.ts-snapshots/split-view-closed-light-mode-linux.png is excluded by !**/*.png
  • tools/snapshot-tests/tests/split-screen.spec.ts-snapshots/split-view-opened-dark-mode-linux.png is excluded by !**/*.png
  • tools/snapshot-tests/tests/split-screen.spec.ts-snapshots/version-dropdown-with-compare-dark-mode-linux.png is excluded by !**/*.png
  • tools/snapshot-tests/tests/split-screen.spec.ts-snapshots/version-dropdown-with-compare-light-mode-linux.png is excluded by !**/*.png

CodeRabbit blocks several paths by default. You can override this behavior by explicitly including those paths in the path filters. For example, including **/dist/** will override the default block on the dist directory, by removing the pattern from both the lists.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: ae469cca-0c2f-4d9e-a118-facff8f0e2ab

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

This PR updates the snapshot testing infrastructure by broadening artifact coverage patterns in the CI workflow, configuring Playwright screenshot comparison tolerances and animation settings, and refactoring test files to use shared screenshot option constants. Configuration files are updated to exclude test results from version control and linting.

Changes

Cohort / File(s) Summary
CI/CD and Build Configuration
.github/workflows/visual-regression.yml, docker-compose.yml
Updated visual regression workflow to capture snapshots from all *-snapshots directories instead of just basic-snapshots, and added target: test-runner to Docker snapshot test service builds.
Ignore and Linting Configuration
.gitignore, biome.jsonc
Added exclusions for tools/snapshot-tests/test-results/ directory to prevent version control tracking and Biome processing.
Playwright Configuration
tools/snapshot-tests/playwright.config.ts
Configured screenshot comparison tolerance with maxDiffPixelRatio: 0.02 and disabled animations for consistent test results.
Test Refactoring
tools/snapshot-tests/tests/narrow-screen.spec.ts, tools/snapshot-tests/tests/split-screen.spec.ts
Introduced shared screenshotOpts constant with { fullPage: true } and replaced repeated inline option objects across all toHaveScreenshot() calls.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Possibly related PRs

  • Adding snapshot tests #321 — Modifies the same snapshot testing infrastructure files (playwright.config.ts, docker-compose.yml, .github/workflows/visual-regression.yml) with related artifact handling changes.

Poem

🐰 Snapshots refined with tolerant eyes,
Test results hidden from git's surprise,
Screenshots shared through constants so neat,
Animations disabled make tests complete! 📸✨

🚥 Pre-merge checks | ✅ 1 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Title check ❓ Inconclusive The title 'Fix/visual regression tests' is vague and uses a generic prefix ('Fix/') that doesn't clearly convey the main purpose of the changes. Revise the title to be more descriptive of the actual changes, such as 'Update visual regression test configuration and snapshots' or 'Improve snapshot testing setup for multiple test suites'.
✅ Passed checks (1 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/visual-regression-tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions
Copy link
Copy Markdown
Contributor

Visual Regression Test Report ✅ Passed

Github run id: 23740502952

🔗 Artifacts: Download

@tomusdrw tomusdrw merged commit 0b489db into main Mar 30, 2026
5 checks passed
@tomusdrw tomusdrw deleted the fix/visual-regression-tests branch March 30, 2026 10:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant