[3.1] add a final job that passes if all tests pass making it easier to branch protect on checks #56
Conversation
.github/workflows/build.yaml
Outdated
|
|
||
| all-passing: | ||
| name: All Required Tests Passed | ||
| needs: [tests, lr-tests, np-tests] |
There was a problem hiding this comment.
Since lr-tests are not always run, does this work?
There was a problem hiding this comment.
I think so, because
A default status check of
success()is applied
and success() is defined as
Returns
truewhen none of the previous steps have failed or been canceled.
So lr-tests being skipped means it is neither failed nor cancelled.
Similarly, all-passing will be skipped if dependent jobs fail, and afaict the branch protection rule does require the test to pass -- skipped does not meet the requirement.
There was a problem hiding this comment.
welp, I guess not 😂
Frustratingly I checked a simple workflow in a private repo beforehand but apparently some combination I didn't account for.
There was a problem hiding this comment.
Lol, I noticed the test run and saw it passed, so approved it.
There was a problem hiding this comment.
Similarly, all-passing will be skipped if dependent jobs fail, and afaict the branch protection rule does require the test to pass -- skipped does not meet the requirement.
and bizarrely that seems false as well
Currently PRs can't be merged because they aren't satisfying branch protection rules. Turns out that a branch protection rule specifying, for example, "Tests" doesn't satisfy the matrix of tests such as "Tests (ubuntu18)", "Tests (ubuntu20)", etc
Making the branch protection rule specify all matrix'ed jobs is too brittle: the names of jobs will change overtime as we add and remove new tests, such as an upcoming test that checks the native contract unit tests.
Instead, let's create a final job in the workflow that only passes if all dependent jobs pass -- where all dependent jobs are all the test jobs. That way the branch protection rule can simply check that "All Required Tests Passed" passed and the workflow can evolve over time without needing to touch the branch protection rule.