Using @Retry with SETUP_FEATURE_CLEANUP mode incorrectly reports failed tests for unrolled features

[rieske]

Issue description

Using @Retry with Retry.Mode.SETUP_FEATURE_CLEANUP incorrectly reports failed tests for unrolled features.
An iteration failure is wrongly attributed to the subsequent iteration in the test report.

How to reproduce

@spock.lang.Retry(mode = spock.lang.Retry.Mode.SETUP_FEATURE_CLEANUP)
class RetrySpec extends spock.lang.Specification {
    def "retried feature"() {
        expect:
        false

        where:
        foo << ['a', 'b', 'c', 'd']
    }
}

will yield:

The failure of the fourth iteration seems to be attributed to the feature method as seen in the test report:

Without the Retry annotation, or when using Retry.Mode.ITERATION, the test failures are reported as expected:


spock.lang.Retry.Mode.SETUP_FEATURE_CLEANUP also does not show this problem with org.spockframework:spock-core:1.3-groovy-2.5

Another case demonstrating the problem:

@spock.lang.Retry(mode = spock.lang.Retry.Mode.SETUP_FEATURE_CLEANUP)
class RetrySpec extends spock.lang.Specification {
    def "confusing case"() {
        expect:
        result

        where:
        result << [true, false]
    }
}

In this case, only the second iteration is made to fail and the report looks like this:

Flipping the where clause to result << [false, true] yields:

Additional Environment information

I used Gradle runner in IntelliJ and Gradle test reports.
I also tried it with a Maven project and using IntelliJ's test runner the reports showed the same problem.

Build-tool dependencies used

testImplementation("org.spockframework:spock-core:2.0-M4-groovy-3.0")

[leonard84]

Seems we have overlooked a case, we already removed Retry.Mode.FEATURE as it doesn't work properly with the changed reporting in JUnit Platform.