Retry Spock2 parameterized methods independently of the class #202
Conversation
pshevche
requested review from
marcphilipp,
skuzzle,
leonard84,
jean-andre-gauthier and
snoopcheri
| @Override | ||
| public TestFramework createRetrying(TestFrameworkTemplate template, TestFramework testFramework, TestNames failedTests) { | ||
| DefaultTestFilter failedTestsFilter = testFilterFor(failedTests, false, template); | ||
| DefaultTestFilter failedTestsFilter = testFilterFor(failedTests, isSpock2Used, template); |
There was a problem hiding this comment.
💬 This boolean will be propagated ascanRunParameterizedSpockMethods to https://github.com/gradle/test-retry-gradle-plugin/blob/main/plugin/src/main/java/org/gradle/testretry/internal/executer/framework/BaseJunitTestFrameworkStrategy.java#L91 that ultimately decides how to retry parameterized test failures: as a whole class or individually.
pshevche
force-pushed
the
pshevche/retry_spock2_parameterized_methods
branch
from
d7f6f53
to
ade7b52
Compare
| } else if (testFramework instanceof TestNGTestFramework) { | ||
| return new TestNgTestFrameworkStrategy(); | ||
| } else { | ||
| return null; | ||
| } | ||
| } | ||
|
|
||
| static boolean isSpock2Used(JvmTestExecutionSpec spec) { | ||
| Iterator<? extends File> classpathIterator = spec.getClasspath().iterator(); |
There was a problem hiding this comment.
❓ Don't we also have to check spec.getModulePath() since Spock 2 should end up there if java.modularity.inferModulePath is set to true in the build script?
There was a problem hiding this comment.
It seems like setting it to true is not enough. Does it have to be a similar setup as described in https://docs.gradle.org/current/samples/sample_java_modules_multi_project_with_integration_tests.html? Where the test source set is a module itself?
There was a problem hiding this comment.
I added a check of module path in d06b375. But I have not managed to set up a test yet where that would happen. I can look into it once I am back.
Signed-off-by: Pavlo Shevchenko <pshevchenko@gradle.com>
…m suffix Signed-off-by: Pavlo Shevchenko <pshevchenko@gradle.com>
…etried as a whole Signed-off-by: Pavlo Shevchenko <pshevchenko@gradle.com>
pshevche
force-pushed
the
pshevche/retry_spock2_parameterized_methods
branch
from
d06b375
to
9661e8d
Compare
...in/src/main/java/org/gradle/testretry/internal/executer/framework/TestFrameworkStrategy.java
Outdated
Show resolved
Hide resolved
...in/src/main/java/org/gradle/testretry/internal/executer/framework/TestFrameworkStrategy.java
Outdated
Show resolved
Hide resolved
Signed-off-by: Pavlo Shevchenko <pshevchenko@gradle.com>
pshevche
force-pushed
the
pshevche/retry_spock2_parameterized_methods
branch
from
9661e8d
to
e03e80d
Compare
Summary
Previously, if Spock tests were executed using Gradle's
JUnitPlatformTestFramework, we would retry an entire class if one of its parameterized methods failed. This applied both to Spock 2 tests and Spock 1 tests run with JUnit Vintage engine. However, parameterized tests cannot be retried independently only in the latter case.This PR changes that and checks for
spock-core-2*JAR on the classpath to differentiate between both cases. If the JAR is present, parameterized tests will be retried independently, otherwise, as an entire class.