feat(ci): on-demand golden regenerate workflow#577
Merged
Conversation
Permanent workflow_dispatch workflow that regenerates the visual- regression baselines on the dfx01 self-hosted runner and commits them back to the dispatched branch as github-actions[bot]. Replaces the previous pattern of introducing and removing a temporary golden-bootstrap.yaml per regen cycle. Usage: gh workflow run golden-regenerate.yaml --ref <feature-branch> Setup steps mirror the golden-tests job in pull-request.yaml so the regenerated baselines render under the exact toolchain that validates them. Concurrency group golden-regenerate-<ref> with cancel-in-progress keeps two parallel dispatches on the same ref from racing. On a protected ref (develop, main) the push fails by design — no force-push, no bypass. The regenerated PNGs are still uploaded as a golden-baselines artifact so they can be rsynced onto a feature branch. docs/visual-regression-tests.md replaces the bootstrap/download/rsync section with the one-command flow. docs/handbook/README.md picks up the same change for handbook screenshot regeneration.
- golden-regenerate.yaml: narrow `git add` to `test/goldens/screens/*/goldens/` so alchemist's transient `failures/` dirs never accidentally land in a bot-pushed commit (MINOR 3 from review). - golden-regenerate.yaml: tighten fallback artifact `if-no-files-found` from `warn` to `error`. An empty fallback artifact is worse than no artifact at all — the user expects the artifact precisely because the push failed; a silent empty is a footgun (MINOR 5). - docs/visual-regression-tests.md: rewrite the stale intro that still claimed "5 screens, 8 baseline PNGs" (left over from PR #541's pilot description) → 57 page files / 68 Golden PNGs, validated by the required `Visual Regression` check (MINOR 4). - docs/visual-regression-tests.md: extend the "On a protected ref the push fails by design" paragraph to also cover the parallel-human-push / non-fast-forward race — same artifact-fallback path, same recovery (MAJOR 1).
Round-2 review caught that `test/goldens/screens/*/goldens/` (the narrow pattern from round 1) silently matches zero files — git pathspecs are not shell globs, the trailing `/goldens/` anchors a directory exactly and `*` does not expand through it. Result: every workflow_dispatch run would have aborted at `git add` with `fatal: pathspec ... did not match any files` before reaching the no-diff early-exit. Regression of the round-1 narrow-add fix. Switch to `'test/goldens/screens/**/goldens/**'` — verified locally: matches the same files as the artifact path on line 92 and excludes alchemist's transient `failures/` directories (alchemist writes them one level up at `<feature>/failures/...`, not inside `goldens/`). Also: rewrite stale "Pilot scope" section in visual-regression-tests.md that still listed only the 5 pilot screens — replaced with a generic "Layout" section describing the per-feature directory convention, which is now accurate at 57 page files / 68 PNGs.
Final stale-pilot reference on docs/visual-regression-tests.md:80 caught by subagent round-3 review. Was inside the "Adding a new golden test" how-to bullet, not a scope claim, but the term "pilot" is no longer accurate now that every page has a Golden.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Permanent `workflow_dispatch`-only workflow `.github/workflows/golden-regenerate.yaml` that regenerates visual-regression baselines on the dfx01 self-hosted runner and commits them back to the dispatched branch as `github-actions[bot]`.
Replaces the previous pattern of introducing and removing a temporary `golden-bootstrap.yaml` per regen cycle (documented in `docs/visual-regression-tests.md` until this PR).
Trigger
```bash
gh workflow run golden-regenerate.yaml --ref
```
No inputs needed — the workflow uses `github.ref` as the checkout ref.
Behaviour
Failure mode on protected branches
`develop` and `main` are protected by ruleset. A dispatch against either fails cleanly on the `git push` step — no force-push, no bypass. To still recover the regen output, the workflow uploads the regenerated PNGs as a `golden-baselines` artifact whenever the push step fails; rsync them onto a feature branch and commit there.
Doc updates
Out of scope
Verification