Skip to content

revert: retry a hung visual shard on the label and scheduled visual runs - #3107

Merged
mfal merged 2 commits into
mainfrom
revert/visual-shard-retry-label-3090
Sep 2, 2026
Merged

revert: retry a hung visual shard on the label and scheduled visual runs#3107
mfal merged 2 commits into
mainfrom
revert/visual-shard-retry-label-3090

Conversation

@mfal

@mfal mfal commented Sep 2, 2026

Copy link
Copy Markdown
Member

Reverts #3090, and removes the same retry from the scheduled run.

The premise did not hold. #3090 read the whole-shard failure — every test in one shard reporting Could not capture a stable screenshot within 5000ms, 63 of 64 blocks on firefox-linux — as a hung browser process, and gave the label path three attempts against it.

The first real exercise of that retry was run 33633352648 on #3104. It ran all three attempts: 118 of 120 tests failed each time, in both browsers, 1110s in total. Shard 1 of the same PR's test.yml run did the same — 85 of 86 tests, at 474s, 472s and 472s. The attempts share the runner, so a machine that produces this failure produces it three times.

So the retry buys nothing on the observed evidence, and it costs the label path 15 minutes of timeout-minutes plus up to 3× the runner time whenever a diff is genuine — the case where a reviewer wants the diff artifacts quickly.

Two commits, because only one of them is a revert

Revert #3090attempts: "3" and timeout-minutes: 45 off the label path, back to 30. A plain git revert, one file.

The scheduled run is not #3090's doing, so removing its retry is a change rather than a revert. It is where the retry started: three attempts as a step-level loop over the whole unsharded suite, from before #3006 sharded it, on the same firefox-hang reasoning. Same evidence applies, so it goes too. Nightly runner time is cheaper than a PR's, but three identical failures buy nothing at any price, and the red they report is unchanged.

Its timeout goes 45 → 30 as well, the label path's number for this same shape at one attempt. 45 was sized for the unsharded job, where a single attempt took 24-34 minutes.

What stays

attempts stays on .github/actions/run-visual-shard, now with no caller. It documents a real mechanism for a hang that shows up on its own, and deleting the loop would reach past this revert into #3006. Its description now names the failure it does not cover and points at #3106. Say the word and I strip it.

What remains open

The failure itself is #3106, with the five main runs it hit and the lead #3090 already noted: toMatchScreenshot runs on vitest's default 5000ms stability budget, while waitForPaintedContent right beside it deliberately carries 20s for slower CI hardware.

After this, nothing retries a visual shard anywhere, so #3106 reports as a plain red shard with no diff artifacts until it is fixed. #3104, which wired the same retry into test.yml, is closed for the same reason.

related #3090, #3106

🤖 Generated with Claude Code

This reverts commit 2aed064.

The premise did not hold. #3090 read the whole-shard failure — every test in
one shard reporting "Could not capture a stable screenshot within 5000ms", 63
of 64 blocks on firefox-linux — as a hung browser process, and gave the label
path three attempts against it.

Its first real exercise, run 33633352648 on #3104, ran all three: 118 of 120
tests failed each time, in both browsers, at 1110s total. Shard 1 of that PR's
`test.yml` run did the same, 85 of 86 tests over 474s, 472s and 472s. The
attempts share the runner, so a machine that produces this failure produces it
three times.

So the retry buys nothing here and costs the label path 15 minutes of timeout
and up to 3x the runner time on a genuine diff. The failure itself is #3106,
where the stability budget #3090 already flagged is the lead.

The scheduled run keeps `attempts: "3"` from #3006 — untouched here, because
this reverts one PR and that one predates it. It is worth the same question.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Coverage Report for ./packages/components/

Status Category Percentage Covered / Total
🔵 Lines 78.69% 746 / 948
🔵 Statements 78.57% 763 / 971
🔵 Functions 80.09% 165 / 206
🔵 Branches 70.33% 377 / 536
File CoverageNo changed files found.
Generated in workflow #6596 for commit 2a0b656 by the Vitest Coverage Report Action

Same evidence, and this is where the retry started. The three attempts predate
the sharding of #3006 — they were a step-level loop over the whole unsharded
suite, on the same reasoning #3090 later carried to the label path: a hung
firefox process needs a fresh browser, which vitest's per-test `retry` cannot
give it.

The failure that actually shows up does not need a fresh browser, it needs a
different runner. Run 33633352648 ran all three attempts of a shard to 118 of
120 failed tests, in both browsers, at 1110s; the `test.yml` shard beside it did
85 of 86 over 474s, 472s and 472s. Nightly runner time is cheaper than a PR's,
but three identical failures buy nothing at any price, and the red they report
is the same either way.

Timeout 45 -> 30, the label path's number for this same shape at one attempt.
45 was sized for the unsharded job, where one attempt alone took 24-34 minutes.

`attempts` stays on the action with no caller. It documents a real mechanism for
a hang that shows up on its own, and deleting it would reach past this revert
into #3006. Its description now says the failure it does not cover, and points
at #3106.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@mfal mfal changed the title revert: retry a hung visual shard on the label workflow revert: retry a hung visual shard on the label and scheduled visual runs Sep 2, 2026
@github-actions

github-actions Bot commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

🚀 Preview Deployment

Preview environments are ready:

Type URL
docs pr-3107.docs.review.flow-components.de
storybook pr-3107.storybook.review.flow-components.de

Images:

  • docs: ghcr.io/mittwald/flow/docs:pr-3107
  • storybook: ghcr.io/mittwald/flow/storybook:pr-3107

@mfal
mfal merged commit 8913802 into main Sep 2, 2026
24 of 25 checks passed
@mfal
mfal deleted the revert/visual-shard-retry-label-3090 branch September 2, 2026 14:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants