macOS x64 browser tests should consistently pass in CI without retrying the entire testing stage #38988
Labels
ci-concern
dev-concern
OS/macOS
priority/P2
A bad problem. We might uplift this to the next planned release.
Description
Browser tests have their own built-in retry mechanism, but this is not sufficient on its own. We had to rebuild often enough, that we ended up adding an automated retry in CI for the entire browser test stage, including a notification whenever it's triggered. In the last 3 months we got ~160 such notifications (Slack search:
"test-browser" "was retried" "times on" in:#devops-bot-dev from:@Jenkins CI
) - that's just from the public pipeline.Each retry extends the build time and increases the size of the build queue for macOS/iOS nodes, causing everyone on the dev/qa teams to wait longer.
Those tests need to consistently pass without a retry for the entire testing stage.
PS. We just added the same retry for network-audit on macos x64, for the same reason.
The text was updated successfully, but these errors were encountered: