Groovy 3 Unit Test Class: TestNGException at Parameters.java:437

[ctzen]

TestNG Version

Note: only the latest version is supported

7.3.0

Expected behavior

Test pass without exception

Actual behavior

Gradle suite > Gradle test > foo.FooTest > setMetaClass FAILED
org.testng.TestNGException at Parameters.java:437
Received a failure event for test with unknown id '55.7'. Registered test ids: '[55.3, 55.2, 55.1, :test]'

Is the issue reproductible on runner?

• Shell
• Maven
• Gradle
• Ant
• Eclipse
• IntelliJ
• NetBeans

Test case sample

Please, share the test case (as small as possible) which shows the issue

Java version AdoptOpenJDK (build 25.262-b10, mixed mode)
Gradle version 6.6.1
Groovy version 3.0.5

Project attached: g3tng.tar.gz

src/test/groovy/foo/FooTest.groovy

package foo
import groovy.transform.CompileStatic
import org.testng.annotations.Test
@CompileStatic
@Test
class FooTest {
    void test() {
        assert true
    }
}

build.gradle.kts

plugins {
    groovy
}
repositories {
    jcenter()
}
dependencies {
    implementation("org.codehaus.groovy:groovy-all:3.0.5")
    testImplementation("org.testng:testng:7.3.0")
}
tasks.test {
    useTestNG()
}

Observations:

1. No issue with groovy version 2.x

2. TestNG selects setMetaClass and setProperty as test methods.
   When runs with ReportNG:

org.testng.TestNGException:
Cannot inject @Test annotated Method [setMetaClass] with [interface groovy.lang.MetaClass].

org.testng.TestNGException:
Cannot inject @Test annotated Method [setProperty] with [class java.lang.String, class java.lang.Object].

[ctzen]

Found a workaround by filtering:

build.gradle.kts

plugins {
    groovy
}
repositories {
    jcenter()
}
dependencies {
    implementation("org.codehaus.groovy:groovy-all:3.0.5")
    testImplementation("org.testng:testng:7.3.0")
}
tasks.test {
    useTestNG()
    filter {
        excludeTestsMatching("*.setMetaClass")
        excludeTestsMatching("*.setProperty")
    }
}

[ctzen]

https://issues.apache.org/jira/browse/GROOVY-9705?focusedCommentId=17187767&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17187767

Yes, we changed some time back the mechanism to mark internal methods like setMetaClass and setProperty. We used to make them synthetic but that interfered with Java integration. We now annotate them with @internal. It would be good for that to be taken up by TestNG like it has for Jacoco code coverage and other tools. See GROOVY-8802 and GROOVY-8495.

[krmahadevan]

@ctzen - Can you please help me understand what is expected from TestNG in this case ? I am not that familiar with Groovy.

[ctzen]

I believe the Groovy project people can explain it better than me, perhaps you can ask in that JIRA.

However, this is my understanding of the issue.

Groovy augment groovy classes with internal methods when converting them to Java classes.
2 of such augmented methods are setMetaClass() and setProperty().

Before Groovy 3, these internal methods were marked as synthetic, which TestNG would ignore when discovering test methods. (I traced the TestNG method discovery to AnnotationHelper.findMethodsWithAnnotation())

Since Groovy 3, these internal methods are no longer marked as synthetic, and TestNG will pick them up as test methods.
These internal methods are now annotated with @groovy.transform.Internal

The fix is for TestNG to NOT pick these Groovy internal methods up as test methods.

The straight forward way is of course to ignored methods annotated with @groovy.transform.Internal.
However, wouldn't that means Groovy will become a transitional dependency of TestNG?

Another possible way is to enhance the TestNG test method discovery. The 2 erroneous methods both have parameters but no DataProvider. If TestNG is to ignore methods with parameters but no DataProvider, that should also solve this issue.

May be we can ask the JACOCO project how they did it?

[krmahadevan]

@ctzen - You kind of explained the problem in a very clear way. Thank you so much. Now I understand the issue.

@juherr @cbeust - What are your thoughts on how to go about fixing this?

[juherr]

@krmahadevan We already have some Groovy tests. Do. you reproduce the issue?

[krmahadevan]

@juherr - we dont have any tests breaking as part of our build. I think the problem comes only when we move to groovy 3.

[juherr]

For the record, some related Jacoco issues: jacoco/jacoco#610 jacoco/jacoco#733