Separate stable and unstable E2E tests for CI run #12059
Labels
a-Build
CI, task automation, build pipeline
a-Testing
Testing-related traits such as efficiency, robustness, coverage
c.Task
Other non-user-facing works, e.g. refactoring, adding tests
help wanted
Moderate difficulty, small localized change; suitable for novice contributors
As mentioned in #12035, E2E tests instability is a perpetual problem that has plagued us.
However, it is noteworthy that only a small number of E2E tests are actually unstable enough to warrant rerunning up to 5 times (and sometimes still failing after that). This means that re-running all E2E tests just to have those few unstable tests passing is often time wasted.
Proposal
To have a separate GitHub Actions job (not a separate workflow) for unstable E2E tests.
The text was updated successfully, but these errors were encountered: