ci: disable fail-fast for Rust version matrix jobs #640
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Tower has test and check jobs on CI that run on a build matrix including a number of Rust versions. By default, GitHub Actions has fail-fast semantics for matrix jobs, so if any matrix job fails, the rest are cancelled and the build is failed. This is intended to help builds complete faster. This isn't really the ideal behavior for testing across multiple Rust versions. When a build fails on a particular toolchain version, we would ideally like to know whether the failure is localized to that version or exists on _all_ Rust versions. This is particularly important for builds on nightly Rust, as the nightly toolchain is more likely to contain compiler regressions that might not be our fault at all. Similarly, we might want to know if a change only broke the build on our MSRV, or if it broke the build everywhere --- such an issue would be fixed differently. Therefore, this PR changes the CI workflow to disable fail-fast behavior on the cross-version test jobs. This way, we will always see the results of the jobs for each Rust toolchain version. This does *not* change what jobs are required to merge a PR --- all versions must still pass (which we may also want to change, but that's a separate issue).
davidpdrsn
approved these changes
Feb 11, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Tower has test and check jobs on CI that run on a build matrix
including a number of Rust versions. By default, GitHub Actions has
fail-fast semantics for matrix jobs, so if any matrix job fails, the
rest are cancelled and the build is failed. This is intended to help
builds complete faster.
This isn't really the ideal behavior for testing across multiple Rust
versions. When a build fails on a particular toolchain version, we would
ideally like to know whether the failure is localized to that version or
exists on all Rust versions. This is particularly important for builds
on nightly Rust, as the nightly toolchain is more likely to contain
compiler regressions that might not be our fault at all. Similarly, we
might want to know if a change only broke the build on our MSRV, or if
it broke the build everywhere --- such an issue would be fixed
differently.
This also currently means that the nightly test run failing will prevent
PRs from being merged, even if the failure is due to a nightly compiler
regression. We currently only require the stable and MSRV test runs
to pass in order to merge a PR, but because the fail-fast behavior
will cancel them if the nightly build fails, this means that nightly failing
will effectively prevent merging PRs...which, given that it's not marked
as required, seems different from what we intended.
Therefore, this PR changes the CI workflow to disable fail-fast behavior
on the cross-version test jobs.