[Maven-Surefire] Surefire runs JUnit tests ignore method filtering in case TestNG in classpath

[baev]

In case TestNG in project classpath surefire-maven-plugin runs the JUnit tests ignore method name filtering.

There is a sample project to reproduce: https://github.com/baev/surefire-testng-junit-bug

Simply clone it and run mvn clean test -Dtest=MyTest#asdasdasd

There is no test named asdasdasd and there is no TestNG suites or tests. There is only the dependency in classpath.

TestNG 6.9.10
maven-surefire-plugin 2.19.1

See for more details https://issues.apache.org/jira/browse/SUREFIRE-1225?focusedCommentId=15141199&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15141199

Summon @Tibor17

[Tibor17]

Surefire is using the MethodSelector the same for option junit=true|false.
These are our calls. As @baev found out the selector is not applied to JUnit if the option is set to true.

xmlTest.getMethodSelectors().add( selector );
...
testng.setXmlSuites( xmlSuites );
testng.run();

[Tibor17]

I checked several versions and it looks like only class filter works.

[juherr]

Thanks for the report.

As I understand, XmlMethodSelector only accepts class: https://github.com/cbeust/testng/blob/master/src/main/java/org/testng/xml/XmlMethodSelector.java#L18
But the TestNG CLI is able to filter methods: https://github.com/cbeust/testng/blob/master/src/main/java/org/testng/TestNG.java#L521-L557

@Tibor17
Maybe Surefire can use XmlClass and XmlInclude instead of XmlMethodSelector.
What do you think? If you want, I can try it.

[Tibor17]

Hi @juherr , thx for fast reply.
I appreciate if you make a try. It's TestNGDirectoryTestSuite class.
The Parameterized is more complex runner, so I tried to test similar scenario
mvn test -Dtest=MyTest2#testWithParameters with pure JUnit test class and two methods but still the bad result : either nothing to run or running both methods.

[juherr]

Well, forget what I said about XmlMethodSelector, I was wrong and I confirm it is a TestNG issue: IMethodSelector (first time I see it :p) is not used by JUnit runners.