GH Actions: fail the build if a test run fails #290
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@jrfnl could you please update this pr? |
jrfnl
force-pushed
the
ghactions-fail-on-failing-test
branch
from
95d40e5
to
9f81c8d
Compare
The way things were set up now in the `phpunit` job, no matter whether tests passed or failed, the workflow would always continue. I suspect this may have been set-up this way to make sure that all variations of test runs will actually be run ? The downside is that, while you will see a ❌ for the individual build in the workflow summary, the workflow will not be marked as failed, nor will the individual test builds be marked as failed. If we look back at the last time the workflow was run completely, this can be seen in the annotations below the summary, which show that the Windows test runs all failed. https://github.com/phpDocumentor/ReflectionDocBlock/actions/runs/370378454 I'm proposing to change this now by: * Removing the `continue-on-error` for the test run. * Adding the `fail-fast` key and setting it to `false`. By default this key is set to `true`, which means that if any individual build within the job fails, all other builds within the job will be cancelled. By setting it to `false`, all builds in the matrix will still be run, but if any of them fail, the workflow will be marked as "failed".
jrfnl
force-pushed
the
ghactions-fail-on-failing-test
branch
from
9f81c8d
to
63aa020
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The way things were set up now in the
phpunitjob, no matter whether tests passed or failed, the workflow would always continue.I suspect this may have been set-up this way to make sure that all variations of test runs will actually be run ?
The downside is that, while you will see a ❌ for the individual build in the workflow summary, the workflow will not be marked as failed, nor will the individual test builds be marked as failed.
If we look back at the last time the workflow was run completely, this can be seen in the annotations below the summary, which show that the Windows test runs all failed.
https://github.com/phpDocumentor/ReflectionDocBlock/actions/runs/370378454
I'm proposing to change this now by:
continue-on-errorfor the test run.fail-fastkey and setting it tofalse.By default this key is set to
true, which means that if any individual build within the job fails, all other builds within the job will be cancelled.By setting it to
false, all builds in the matrix will still be run, but if any of them fail, the workflow will be marked as "failed".Opening as draft to expose issue #287 properly.