Skip to content

test: refresh blog visual regression baselines (macOS + Linux)#329

Merged
pftg merged 2 commits into
masterfrom
chore/refresh-blog-baselines
May 8, 2026
Merged

test: refresh blog visual regression baselines (macOS + Linux)#329
pftg merged 2 commits into
masterfrom
chore/refresh-blog-baselines

Conversation

@pftg
Copy link
Copy Markdown
Member

@pftg pftg commented May 8, 2026

Summary

Refreshes the 3 stale blog visual regression baselines on both macOS and Linux that have been failing on master with identical `difference_level: 0.062389564043209875`. Verified on both runners: all tests pass against new baselines.

Root cause

Baselines last captured 2026-04-14. Since then 5 intentional blog UI changes shipped without updating baselines:

Date Commit Change
2026-04-30 `a7c5cd3e` fix(blog): index card tag taxonomy + ICP hero copy
2026-05-02 `566044d1` Compact reading sections + 6 post engagement improvements (#311)
2026-05-03 `b4d6d1e9` feat(blog): centered single-column typography (post + index)
2026-05-04 `f1aeb21b` Use explicit `.post-description` class for blog listing description (#320)
2026-05-05 `059157c4` fix(blog): show descriptions on mobile + fix overflow at narrow viewports

Each shipped while the visual regression tests were already failing — the gate fired but nobody updated baselines. The 0.062 difference is the cumulative drift, not a single regression.

Investigation path

  1. Reproduced on master — same exact `difference_level` as on PRs seo: consolidate fractional CTO pages (fix position-75 cannibalization) #326, seo: consolidate Rails perf posts into 2026 canonical (hub-and-spoke) #327, seo: new /services/react-development/ page (Block 3) #328. Confirmed pre-existing.
  2. Diff image (`index.diff.png`): yellow boxes around blog post cards (dynamic content), not on layout/CSS. Content drift, not layout drift.
  3. Checked baseline file dates vs blog template change history. Baselines from April 14, 5 intentional changes shipped after.
  4. Examined gem source (`snap_diff-capybara`): baseline pulled via `git show HEAD:`, not from working tree. Working-tree updates do nothing until committed.
  5. Captured new baselines via `FORCE_SCREENSHOT_UPDATE=true`, committed, re-ran tests on the new HEAD — all pass.

What changed

macOS baselines (commit `6321fe9e`):

  • `test/fixtures/screenshots/macos/desktop/blog/index.png`
  • `test/fixtures/screenshots/macos/mobile/blog/index.png`
  • `test/fixtures/screenshots/macos/mobile/blog/post.png`

Linux baselines (commit `918f8668`):

  • `test/fixtures/screenshots/linux/desktop/blog/index.png`
  • `test/fixtures/screenshots/linux/mobile/blog/index.png`
  • `test/fixtures/screenshots/linux/mobile/blog/index/_pagination.png`
  • `test/fixtures/screenshots/linux/mobile/blog/post.png`

Test plan

  • macOS: `bundle exec ruby -Itest test/system/desktop_site_test.rb -n test_blog_index` → 1 run, 0 failures
  • macOS: `bundle exec ruby -Itest test/system/mobile_site_test.rb -n "/test_blog_index|test_blog_post/"` → 3 runs, 0 failures
  • Linux: same commands via Docker (`bin/docked t bundle exec ruby -Itest ...`) → 1 run desktop / 3 runs mobile / 0 failures
  • Verified via fresh `git show HEAD:` lookup that committed baselines match captured screenshots

Process improvement (for separate session)

The visual regression gate is supposed to fire BEFORE merge per CLAUDE.md. The fact that 5 commits shipped through it without anyone updating baselines suggests the gate isn't actually blocking merges. Worth investigating in a follow-up — the rule's intent is to catch drift at the boundary, not 3 weeks later.

🤖 Generated with Claude Code

…ges)

Pre-existing visual regression failures on master:
- DesktopSiteTest#test_blog_index
- MobileSiteTest#test_blog_index
- MobileSiteTest#test_blog_post

All 3 fail with identical difference_level: 0.062389564043209875.

Root cause: baselines last captured 2026-04-14. Since then 5 intentional
blog UI changes shipped without baseline updates:
- 2026-04-30 fix(blog): index card tag taxonomy + ICP hero copy
- 2026-05-02 Compact reading sections + 6 post engagement improvements
- 2026-05-03 feat(blog): centered single-column typography (post + index)
- 2026-05-04 Use explicit .post-description class for blog listing
- 2026-05-05 fix(blog): show descriptions on mobile + fix overflow

The drift is the cumulative effect of these intentional improvements.
Refreshing macOS baselines via FORCE_SCREENSHOT_UPDATE=true.

Linux baselines also need refresh — Docker daemon not available locally.
Will fire on CI; follow-up needed once Docker is running, OR refresh
through CI's FORCE_SCREENSHOT_UPDATE workflow if available.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 8, 2026

Important

Review skipped

Review was skipped due to path filters

⛔ Files ignored due to path filters (7)
  • test/fixtures/screenshots/linux/desktop/blog/index.png is excluded by !**/*.png
  • test/fixtures/screenshots/linux/mobile/blog/index.png is excluded by !**/*.png
  • test/fixtures/screenshots/linux/mobile/blog/index/_pagination.png is excluded by !**/*.png
  • test/fixtures/screenshots/linux/mobile/blog/post.png is excluded by !**/*.png
  • test/fixtures/screenshots/macos/desktop/blog/index.png is excluded by !**/*.png
  • test/fixtures/screenshots/macos/mobile/blog/index.png is excluded by !**/*.png
  • test/fixtures/screenshots/macos/mobile/blog/post.png is excluded by !**/*.png

CodeRabbit blocks several paths by default. You can override this behavior by explicitly including those paths in the path filters. For example, including **/dist/** will override the default block on the dist directory, by removing the pattern from both the lists.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 0e7b3414-2298-4860-92a0-ba4d2ad3bd8e

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch chore/refresh-blog-baselines

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

…mmit)

Same root cause as the macOS refresh in 6321fe9 — Linux baselines also
went stale through the same 5 intentional blog UI changes between
2026-04-14 and 2026-05-05.

Captured via: docker compose run --env FORCE_SCREENSHOT_UPDATE=true t
bundle exec ruby -Itest test/system/{desktop,mobile}_site_test.rb -n
"/test_blog_index|test_blog_post/"

Verified locally on macOS host (commit 6321fe9) and Linux Docker.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@pftg pftg changed the title test: refresh macOS blog baselines (catch up 3 weeks of intentional UI changes) test: refresh blog visual regression baselines (macOS + Linux) May 8, 2026
@pftg pftg merged commit c94afaf into master May 8, 2026
2 checks passed
@pftg pftg deleted the chore/refresh-blog-baselines branch May 8, 2026 21:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant