NPE thrown by Surefire

[bartoszmajsak]

Hi guys,

once in a while (pretty much randomly, might be race condition) Surefire throws this:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test (default-test) on project trio-services: Execution default-test of goal org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test failed: There was an error in the forked process
[ERROR] org.apache.maven.surefire.testset.TestSetFailedException: java.lang.NullPointerException
[ERROR] at org.apache.maven.surefire.common.junit4.JUnit4RunListener.rethrowAnyTestMechanismFailures(JUnit4RunListener.java:213)
[ERROR] at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun(JUnitCoreWrapper.java:109)
[ERROR] at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.executeEager(JUnitCoreWrapper.java:78)
[ERROR] at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:54)
[ERROR] at org.apache.maven.surefire.junitcore.JUnitCoreProvider.invoke(JUnitCoreProvider.java:144)
[ERROR] at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203)
[ERROR] at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)
[ERROR] at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
[ERROR] Caused by: java.lang.NullPointerException
[ERROR] at org.apache.maven.surefire.common.junit4.JUnit4RunListener.extractClassName(JUnit4RunListener.java:182)
[ERROR] at org.apache.maven.surefire.common.junit4.JUnit4RunListener.getClassName(JUnit4RunListener.java:158)
[ERROR] at org.apache.maven.surefire.common.junit4.JUnit4RunListener.createReportEntry(JUnit4RunListener.java:153)
[ERROR] at org.apache.maven.surefire.common.junit4.JUnit4RunListener.testStarted(JUnit4RunListener.java:91)
[ERROR] at org.junit.runner.notification.SynchronizedRunListener.testStarted(SynchronizedRunListener.java:49)
[ERROR] at org.junit.runner.notification.RunNotifier$3.notifyListener(RunNotifier.java:121)
[ERROR] at org.junit.runner.notification.RunNotifier$SafeNotifier.run(RunNotifier.java:72)
[ERROR] at org.junit.runner.notification.RunNotifier.fireTestStarted(RunNotifier.java:118)
[ERROR] at org.junit.internal.runners.model.EachTestNotifier.fireTestStarted(EachTestNotifier.java:42)
[ERROR] at junitparams.internal.ParameterisedTestMethodRunner.runMethodInvoker(ParameterisedTestMethodRunner.java:45)
[ERROR] at junitparams.internal.ParameterisedTestMethodRunner.runTestMethod(ParameterisedTestMethodRunner.java:40)
[ERROR] at junitparams.internal.ParameterisedTestClassRunner.runParameterisedTest(ParameterisedTestClassRunner.java:147)
[ERROR] at junitparams.JUnitParamsRunner.runChild(JUnitParamsRunner.java:414)
[ERROR] at junitparams.JUnitParamsRunner.runChild(JUnitParamsRunner.java:385)
[ERROR] at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
[ERROR] at org.apache.maven.surefire.junitcore.pc.Scheduler$1.run(Scheduler.java:387)
[ERROR] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
[ERROR] at java.util.concurrent.FutureTask.run(FutureTask.java:262)
[ERROR] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[ERROR] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[ERROR] at java.lang.Thread.run(Thread.java:724)
[ERROR] -> [Help 1]

Here are my environment details:

Apache Maven 3.2.3 (33f8c3e1027c3ddde99d3cdebad2656a31e8fdf4; 2014-08-11T22:58:10+02:00)
Java version: 1.7.0_40, vendor: Oracle Corporation
Java home: /usr/lib/jvm/jdk1.7.0_40/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "3.13.0-37-generic", arch: "amd64", family: "unix"

• JUnit 4.12
• Surefire 2.18.1
• JUnitParams 1.0.4

On top of that I run Surefire tests in parallel with following settings:

• parallel='classesAndMethods'
• perCoreThreadCount=false
• threadCount=8
• useUnlimitedThreads=false
• threadCountSuites=0
• threadCountClasses=0
• threadCountMethods=0
• parallelOptimized=true

Any ideas what might be the problem?

Note: this problem was also present with

• JUnit 4.11
• Surefire 2.18
• JUnitParams 1.0.3

[apierzch]

Thank you for this detailed report. We will investigate it very soon.

[apierzch]

The issue reported by you does not seem to be present after refactoring I've done for version 2.0.
However, during investigation I have noticed that JUnitParams sometimes does not execute all tests when running methods in parallel (which you might notice by looking at "Tests run" output. I strongly advise not to use any of following parallel options with JUnitParams:

• "classesAndMethods"
• "methods"
• "both"

Any other configuration should work without problems, please report if otherwise.

[apierzch]

I have created #35 for the problem above

[bartoszmajsak]

Thanks for the prompt assistance :) Another "workaround" would be to put net.jcip.annotations.NotThreadSafe annotation on the classes where JUnitParams is involved and keep the surefire settings as they are.

[deepakab03]

Thanks @bartoszmajsak for the workaround to use @ NotThreadSafe.
@apierzch not being able to run in parallel is a significant limitation, hope there is something in the roadmap to address this. Running the builds in a non parallel mode results in the build time increasing by 4-5 times which is unacceptable in some cases. Thank you for all your work in this great library.