NullPointerException in jqwik on test failure

[tmohme]

Testing Problem

With one tests with generated nullable parameters, I get a NullPointerException in jqwik when the test fails.

This is the stack trace I get on the console:

java.lang.NullPointerException
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
	at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
	at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
	at net.jqwik.engine.execution.reporting.ParameterChangesDetector.atLeastOneChangedParameterHasEqualsImplementation(ParameterChangesDetector.java:19)
	at net.jqwik.engine.execution.reporting.ParameterChangesDetector.haveParametersChanged(ParameterChangesDetector.java:12)
	at net.jqwik.engine.execution.reporting.ExecutionResultReport.reportParameterChanges(ExecutionResultReport.java:91)
	at net.jqwik.engine.execution.reporting.ExecutionResultReport.lambda$appendSamples$0(ExecutionResultReport.java:65)
	at java.base/java.util.Optional.ifPresent(Optional.java:183)
	at net.jqwik.engine.execution.reporting.ExecutionResultReport.appendSamples(ExecutionResultReport.java:59)
	at net.jqwik.engine.execution.reporting.ExecutionResultReport.buildJqwikReport(ExecutionResultReport.java:49)
	at net.jqwik.engine.execution.reporting.ExecutionResultReport.from(ExecutionResultReport.java:33)
	at net.jqwik.engine.execution.PropertyMethodExecutor.reportResult(PropertyMethodExecutor.java:134)
	at net.jqwik.engine.execution.PropertyMethodExecutor.executePropertyMethod(PropertyMethodExecutor.java:99)
	at net.jqwik.engine.execution.PropertyMethodExecutor.execute(PropertyMethodExecutor.java:42)
	at net.jqwik.engine.execution.PropertyTaskCreator.executeTestMethod(PropertyTaskCreator.java:128)
	at net.jqwik.engine.execution.PropertyTaskCreator.lambda$createTask$1(PropertyTaskCreator.java:55)
	at net.jqwik.engine.execution.pipeline.ExecutionTask$1.lambda$execute$0(ExecutionTask.java:31)
	at net.jqwik.engine.execution.lifecycle.CurrentTestDescriptor.runWithDescriptor(CurrentTestDescriptor.java:27)
	at net.jqwik.engine.execution.pipeline.ExecutionTask$1.execute(ExecutionTask.java:31)
	at net.jqwik.engine.execution.pipeline.ExecutionPipeline.runToTermination(ExecutionPipeline.java:82)
	at net.jqwik.engine.execution.JqwikExecutor.execute(JqwikExecutor.java:46)
	at net.jqwik.engine.JqwikTestEngine.executeTests(JqwikTestEngine.java:77)
	at net.jqwik.engine.JqwikTestEngine.execute(JqwikTestEngine.java:61)

But when I stop the program when the NPE happens (via exception-breakpoint in IntelliJ), I find the following:

• the NPE actually happens in net.jqwik.engine.execution.reporting.ArrayReportingFormat#appliesTo because the actual argument is null
• some levels up in the call hierarchy, in net.jqwik.engine.execution.reporting.SampleReporter#reportParameters, the parameterName is set, while the parameterValue is null

This is the stack trace I get with the exception breakpoint:

appliesTo:10, ArrayReportingFormat (net.jqwik.engine.execution.reporting)
lambda$reportingFormatFinder$3:123, ValueReport (net.jqwik.engine.execution.reporting)
test:-1, 1064197277 (net.jqwik.engine.execution.reporting.ValueReport$$Lambda$509)
accept:176, ReferencePipeline$2$1 (java.util.stream)
tryAdvance:1632, ArrayList$ArrayListSpliterator (java.util)
forEachWithCancel:127, ReferencePipeline (java.util.stream)
copyIntoWithCancel:502, AbstractPipeline (java.util.stream)
copyInto:488, AbstractPipeline (java.util.stream)
wrapAndCopyInto:474, AbstractPipeline (java.util.stream)
evaluateSequential:150, FindOps$FindOp (java.util.stream)
evaluate:234, AbstractPipeline (java.util.stream)
findFirst:543, ReferencePipeline (java.util.stream)
lambda$reportingFormatFinder$4:128, ValueReport (net.jqwik.engine.execution.reporting)
find:-1, 285919215 (net.jqwik.engine.execution.reporting.ValueReport$$Lambda$508)
of:31, ValueReport (net.jqwik.engine.execution.reporting)
of:21, ValueReport (net.jqwik.engine.execution.reporting)
of:16, ValueReport (net.jqwik.engine.execution.reporting)
reportParameters:74, SampleReporter (net.jqwik.engine.execution.reporting)
reportTo:67, SampleReporter (net.jqwik.engine.execution.reporting)
buildReports:57, DefaultReporter (net.jqwik.engine.execution.reporting)
publishReports:50, DefaultReporter (net.jqwik.engine.execution.reporting)
lambda$createFalsifiedSampleReporter$0:202, GenericProperty (net.jqwik.engine.properties)
accept:-1, 734897812 (net.jqwik.engine.properties.GenericProperty$$Lambda$474)
lambda$shrink$1:59, PropertyShrinker (net.jqwik.engine.properties.shrinking)
accept:-1, 976807132 (net.jqwik.engine.properties.shrinking.PropertyShrinker$$Lambda$477)
shrink:72, AbstractSampleShrinker (net.jqwik.engine.properties.shrinking)
shrinkSingleParameter:43, OneAfterTheOtherParameterShrinker (net.jqwik.engine.properties.shrinking)
shrink:25, OneAfterTheOtherParameterShrinker (net.jqwik.engine.properties.shrinking)
shrinkOneParameterAfterTheOther:120, PropertyShrinker (net.jqwik.engine.properties.shrinking)
shrinkAsLongAsSampleImproves:107, PropertyShrinker (net.jqwik.engine.properties.shrinking)
lambda$shrink$3:77, PropertyShrinker (net.jqwik.engine.properties.shrinking)
get:-1, 1075284181 (net.jqwik.engine.properties.shrinking.PropertyShrinker$$Lambda$479)
run$$$capture:1700, CompletableFuture$AsyncSupply (java.util.concurrent)
run:-1, CompletableFuture$AsyncSupply (java.util.concurrent)
 - Async stack trace
<init>:1686, CompletableFuture$AsyncSupply (java.util.concurrent)
asyncSupplyStage:1714, CompletableFuture (java.util.concurrent)
supplyAsync:1915, CompletableFuture (java.util.concurrent)
withTimeout:88, PropertyShrinker (net.jqwik.engine.properties.shrinking)
shrink:81, PropertyShrinker (net.jqwik.engine.properties.shrinking)
shrink:68, PropertyShrinker (net.jqwik.engine.properties.shrinking)
shrink:194, GenericProperty (net.jqwik.engine.properties)
shrinkAndCreateCheckResult:166, GenericProperty (net.jqwik.engine.properties)
check:70, GenericProperty (net.jqwik.engine.properties)
check:58, CheckedProperty (net.jqwik.engine.execution)
executeProperty:124, PropertyMethodExecutor (net.jqwik.engine.execution)
executeMethod:108, PropertyMethodExecutor (net.jqwik.engine.execution)
lambda$executePropertyMethod$0:88, PropertyMethodExecutor (net.jqwik.engine.execution)
lambda$static$0:43, AroundPropertyHook (net.jqwik.api.lifecycle)
lambda$wrap$0:26, HookSupport (net.jqwik.engine.execution.lifecycle)
aroundProperty:70, PropertyDefaults$PropertyDefaultsHook (net.jqwik.api)
lambda$wrap$1:31, HookSupport (net.jqwik.engine.execution.lifecycle)
lambda$wrap$0:26, HookSupport (net.jqwik.engine.execution.lifecycle)
aroundProperty:57, PropertyLifecycleMethodsHook (net.jqwik.engine.hooks.lifecycle)
lambda$wrap$1:31, HookSupport (net.jqwik.engine.execution.lifecycle)
lambda$wrap$0:26, HookSupport (net.jqwik.engine.execution.lifecycle)
aroundProperty:32, StatisticsHook (net.jqwik.engine.hooks.statistics)
lambda$wrap$1:31, HookSupport (net.jqwik.engine.execution.lifecycle)
lambda$wrap$0:26, HookSupport (net.jqwik.engine.execution.lifecycle)
aroundProperty:13, AutoCloseableHook (net.jqwik.engine.hooks.lifecycle)
lambda$wrap$1:31, HookSupport (net.jqwik.engine.execution.lifecycle)
executePropertyMethod:86, PropertyMethodExecutor (net.jqwik.engine.execution)
execute:42, PropertyMethodExecutor (net.jqwik.engine.execution)
executeTestMethod:128, PropertyTaskCreator (net.jqwik.engine.execution)
lambda$createTask$1:55, PropertyTaskCreator (net.jqwik.engine.execution)
lambda$execute$0:31, ExecutionTask$1 (net.jqwik.engine.execution.pipeline)
runWithDescriptor:27, CurrentTestDescriptor (net.jqwik.engine.execution.lifecycle)
execute:31, ExecutionTask$1 (net.jqwik.engine.execution.pipeline)
runToTermination:82, ExecutionPipeline (net.jqwik.engine.execution.pipeline)
execute:46, JqwikExecutor (net.jqwik.engine.execution)
executeTests:77, JqwikTestEngine (net.jqwik.engine)
execute:61, JqwikTestEngine (net.jqwik.engine)
execute:108, EngineExecutionOrchestrator (org.junit.platform.launcher.core)
execute:88, EngineExecutionOrchestrator (org.junit.platform.launcher.core)
lambda$execute$0:54, EngineExecutionOrchestrator (org.junit.platform.launcher.core)
withInterceptedStreams:67, EngineExecutionOrchestrator (org.junit.platform.launcher.core)
execute:52, EngineExecutionOrchestrator (org.junit.platform.launcher.core)
execute:96, DefaultLauncher (org.junit.platform.launcher.core)
execute:75, DefaultLauncher (org.junit.platform.launcher.core)
startRunnerWithArgs:71, JUnit5IdeaTestRunner (com.intellij.junit5)
startRunnerWithArgs:33, IdeaTestRunner$Repeater (com.intellij.rt.junit)
prepareStreamsAndStart:220, JUnitStarter (com.intellij.rt.junit)
main:53, JUnitStarter (com.intellij.rt.junit)

Interestingly, adding a @Report(Reporting.FALSIFIED) annotation to the test produces the expected output and has no problem with the (test-) parameter in question being null.
All this happens with jqwik 1.3.8 and Java 11.

Suggested Solution

We should not get a NullPointerException.

Discussion

If you need more information (like a reproducer) please let me know.

[jlink]

Thanks for the catch. I’ll have a look at it.

[jlink]

Fixed in 7fb5b64

Available in 1.3.9-SNAPSHOT

[tmohme]

Fyi: Just want to confirm that 1.3.9-SNAPSHOT solves my problem.
Thank you for the super fast reaction!