Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Add release blocking job criteria #346
The Kubernetes community has developed an over-reliance on e2e tests that are flaky, slow, and seemingly unmaintained. We, the release team, end up spending much of our time rationalizing and explaining away failing or flaky tests as red-but-green, or ignoring meaningful failures under the guise of perpetual-failure being the expected normal state.
This results in delays leading up to release, contributes to the reputation of .0 releases being beta-quality at best, and allows regressions to slip through the cracks in patch releases.
This PR proposes release blocking job criteria based on #24 to help us eliminate noise and improve signal
The criteria spelled out here are less strict than the criteria in that issue but this is based on what the jobs look like today.
Once this PR lands I would like to move the release-blocking jobs toward the criteria listed, and refine as described in the TODO's. e.g.:
I plan on using #347 as the umbrella issue for that
Agree. I do not think much is lost in terms of release signal by removing these GKE tests from release-blocking. The results will still be available in dashboards such as google-gke and google-gke-staging for those that wish to pay attention to them.
[APPROVALNOTIFIER] This PR is APPROVED
The full list of commands accepted by this bot can be found here.
The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing