TestLauncher reports test success even if test fails #26195
Comments
1 task
bot-gradle
added a commit
that referenced
this issue
Aug 31, 2023
…r more uniform The TestLauncher calls AbstractTestTask.setIgnoreFailures(true) which makes the build pass even if the test failed. This is an unexpected behavior compared to the BuildLauncher as users do expect build failures. The failures were ignored in order not to stop the execution when a test task failed. At the same time, the client can always run the build with the `--continue` flag and have the same behavior. Fixes #26195 TODO - [x] Mention this in the release notes as a potential breaking change Co-authored-by: Donát Csikós <donat@gradle.com>
bot-gradle
added a commit
that referenced
this issue
Sep 1, 2023
…r more uniform The TestLauncher calls AbstractTestTask.setIgnoreFailures(true) which makes the build pass even if the test failed. This is an unexpected behavior compared to the BuildLauncher as users do expect build failures. The failures were ignored in order not to stop the execution when a test task failed. At the same time, the client can always run the build with the `--continue` flag and have the same behavior. Fixes #26195 TODO - [x] Mention this in the release notes as a potential breaking change Co-authored-by: Donát Csikós <donat@gradle.com>
This was referenced Oct 17, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Current Behavior
If I run the test task using BuildLauncher, I get Task :test failed in DefaultTaskFinishEvent and DefaultTaskFailureResult as the execution result. If I run the test task using TestLauncher, I get Task :test SUCCESS in DefaultTaskFinishEvent and DefaultTaskSuccessResult as the execution result.
Expected Behavior
The test outcome and the reported TAPI events should both fail.
Context (optional)
No response
Steps to Reproduce
reproducer project
Gradle version
Gradle 8.2
Build scan URL (optional)
No response
Your Environment (optional)
No response
The text was updated successfully, but these errors were encountered: