useTestNg() has limited list of command line args which can be passed to TestNg runner

[achahan]

The list of command line arguments supported by TestNg runner is changing. New arguments appear. TestNg plugin supports parameters defined in TestNgOptions class only. Which is not flexible and limits features can be used.

Expected Behavior

It is possible to pass TestNg command line arguments as string key/value pairs.

Current Behavior

Can only set limited number of command line attributes.

Context

It is possible to inject TestNg listeners via ServiceLoader. A feature which allows to disable some of injected implementations was added in latest TestNg versions. I want to pass a command line argument "spiListenersToSkip"; but it's not possible as the argument is not supported by plugin.

[jbartok]

Could you pls. provide an exact setup/reproducer to check against, if we add the missing features?

[achahan]

@jbartok Sure. Will provide.

[achahan]

Please, see example projects here: https://github.com/achahan/spiExample
projA contains implementations for two TestNg listeners: MySuiteListener, MyTestListener.
Above listeners will be wired to projB via Service Loader since projB depends on proA.
So, if one will run "gradle test" from projB root the output will be the following:
On Suite Start
On Start
.........
On Finish
On Suite Finish
Now I want to disable wired "MySuiteListener" so that there were no "On Suite ..." messages. This feature is available from TestNg 7.4. There is a TestNg command line parameter "-spilistenerstoskip" followed by comma-separated list of listeners to skip.
The problem is that gradle testNg plugin does not support above mentioned parameter. So, "test" task can't be used with excluded listener.
However list of listeners to skip can be passed from "JavaExec" based gradle task. See projB/build.gradle/runJavaExec task for example.

So, the issue was created because it might make sense to add custom parameters support to gradle testng plugin. In order not to support each of possible TestNg command line parameters. So that one could do something like example below from gradle test task:
test {
useTestNG() {
arguments = [suite:"suite location", spilistenerstoskip:"com.example.MySuiteListener", :"some value"]
}
}

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. Given the limited bandwidth of the team, it will be automatically closed if no further activity occurs. If you're interested in how we try to keep the backlog in a healthy state, please read our blog post on how we refine our backlog. If you feel this is something you could contribute, please have a look at our Contributor Guide. Thank you for your contribution.

[ov7a]

Sorry for the late reply.

Thank you for your interest in Gradle!

This feature request is in the backlog of the relevant team, but this area of Gradle is currently not a focus one, so it might take a while before it gets implemented.

[itkhanz]

I would like to have this feature added to TestNGOptions class so we we can inject this argument -reporter with value org.testng.reporters.XMLReporter:generateTestResultAttributes=true,generateGroupsAttribute=true to TestNG, so TestNG test task will be executed with these specific options. It will then enable TestNG to produce XML reprots with additional custom attributes.
I have explained this in detail in my ticket:
Ability to inject custom comandline arguments to TestNG is missing

I don't think this will require much effort on Gradle's end to enable this feature to allow users to send custom args to TestNG invocation. This is already supported by TestNG and Maven surefire plugin. It will be also helpful for many other use cases where people are using Gradle with TestNG. Please prioritise this issue if possible.

Existing Gradle DSL is really limited in terms of TestNG usage and it would be really helpful to allow this option for maximum flexibility while using TestNG as test framework with gradle.