Skip to content

fix(ci): retry failed snap test cases instead of skipping flaky tests#1209

Merged
graphite-app[bot] merged 1 commit intomainfrom
ignore-flaky-test
Mar 30, 2026
Merged

fix(ci): retry failed snap test cases instead of skipping flaky tests#1209
graphite-app[bot] merged 1 commit intomainfrom
ignore-flaky-test

Conversation

@fengmk2
Copy link
Copy Markdown
Member

@fengmk2 fengmk2 commented Mar 30, 2026

Summary

  • Revert the ignoredPlatforms skip on create-next-command-library so it runs on all platforms again
  • Add .github/scripts/retry-failed-snap-tests.sh that retries only the specific snap test cases whose snap.txt changed, up to 2 times
  • Both cli-snap-test and cli-e2e-test-musl jobs use the shared script

Test plan

  • CI passes on all platforms (Linux, macOS, Windows)
  • create-next-command-library snap test runs on macOS and Windows
  • If a snap test is flaky, CI retries only that specific case instead of failing immediately

@netlify
Copy link
Copy Markdown

netlify bot commented Mar 30, 2026

Deploy Preview for viteplus-preview canceled.

Name Link
🔨 Latest commit 6910ba8
🔍 Latest deploy log https://app.netlify.com/projects/viteplus-preview/deploys/69ca17b78c8bec0008973877

Copy link
Copy Markdown
Member Author

fengmk2 commented Mar 30, 2026


How to use the Graphite Merge Queue

Add the label auto-merge to this PR to add it to the merge queue.

You must have a Graphite account in order to use the merge queue. Sign up using this link.

An organization admin has enabled the Graphite Merge Queue in this repository.

Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.

This stack of pull requests is managed by Graphite. Learn more about stacking.

@fengmk2 fengmk2 self-assigned this Mar 30, 2026
@fengmk2 fengmk2 marked this pull request as ready for review March 30, 2026 02:21
@graphite-app
Copy link
Copy Markdown

graphite-app bot commented Mar 30, 2026

Merge activity

  • Mar 30, 4:41 AM UTC: fengmk2 added this pull request to the Graphite merge queue.
  • Mar 30, 4:51 AM UTC: The Graphite merge queue couldn't merge this PR because it was not satisfying all requirements (Failed CI: 'done', 'CLI snap test (x86_64-unknown-linux-gnu, 2/3)', 'CLI snap test (x86_64-unknown-linux-gnu, 3/3)').
  • Mar 30, 6:26 AM UTC: fengmk2 added this pull request to the Graphite merge queue.
  • Mar 30, 6:34 AM UTC: Merged by the Graphite merge queue.

graphite-app bot pushed a commit that referenced this pull request Mar 30, 2026
…and Windows (#1209)

Only run this test on Linux to avoid flaky failures.
@graphite-app graphite-app bot force-pushed the ignore-flaky-test branch from 0b09691 to 7feb2e7 Compare March 30, 2026 04:42
Copy link
Copy Markdown
Member Author

fengmk2 commented Mar 30, 2026

I will change it to retry instead of ignore

fengmk2 added a commit that referenced this pull request Mar 30, 2026
…tests

Revert the ignoredPlatforms skip on create-next-command-library and
instead add a shared retry script that re-runs only the specific snap
test cases whose snap.txt changed, up to 2 times.

Closes #1209
@fengmk2 fengmk2 changed the title fix(test): skip flaky create-next-command-library snap test on macOS and Windows fix(ci): retry failed snap test cases instead of skipping flaky tests Mar 30, 2026
fengmk2 added a commit that referenced this pull request Mar 30, 2026
…tests

Revert the ignoredPlatforms skip on create-next-command-library and
instead add a shared retry script that re-runs only the specific snap
test cases whose snap.txt changed, up to 2 times.

Closes #1209
@fengmk2 fengmk2 force-pushed the ignore-flaky-test branch from e27d9a1 to 1a3b903 Compare March 30, 2026 05:58
fengmk2 added a commit that referenced this pull request Mar 30, 2026
…and Windows (#1209)

Only run this test on Linux to avoid flaky failures.
fengmk2 added a commit that referenced this pull request Mar 30, 2026
…tests

Revert the ignoredPlatforms skip on create-next-command-library and
instead add a shared retry script that re-runs only the specific snap
test cases whose snap.txt changed, up to 2 times.

Closes #1209
@fengmk2 fengmk2 force-pushed the ignore-flaky-test branch from 1a3b903 to a039baf Compare March 30, 2026 06:08
…#1209)

## Summary

- Revert the `ignoredPlatforms` skip on `create-next-command-library` so it runs on all platforms again
- Add `.github/scripts/retry-failed-snap-tests.sh` that retries only the specific snap test cases whose `snap.txt` changed, up to 2 times
- Both `cli-snap-test` and `cli-e2e-test-musl` jobs use the shared script

## Test plan

- [x] CI passes on all platforms (Linux, macOS, Windows)
- [x] `create-next-command-library` snap test runs on macOS and Windows
- [x] If a snap test is flaky, CI retries only that specific case instead of failing immediately
@graphite-app graphite-app bot force-pushed the ignore-flaky-test branch from a039baf to 6910ba8 Compare March 30, 2026 06:26
@graphite-app graphite-app bot merged commit 6910ba8 into main Mar 30, 2026
35 checks passed
@graphite-app graphite-app bot deleted the ignore-flaky-test branch March 30, 2026 06:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants