A flaky test is paid for twice: once when the job fails and once on the retry, and it also teaches contributors to re-run rather than read a red check. With the runner budget now a hard constraint, flakiness is a budget problem, not only an annoyance.
Establish which tests are actually flaky from CI history, quarantine them out of the PR path, and fix them behind that. The migrated JIRAs #16740, #16207 and #15248 have partial lists from the Azure Pipelines era.
Part of #19524.
A flaky test is paid for twice: once when the job fails and once on the retry, and it also teaches contributors to re-run rather than read a red check. With the runner budget now a hard constraint, flakiness is a budget problem, not only an annoyance.
Establish which tests are actually flaky from CI history, quarantine them out of the PR path, and fix them behind that. The migrated JIRAs #16740, #16207 and #15248 have partial lists from the Azure Pipelines era.
Part of #19524.