Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NullPointerException in jqwik on test failure #132

Closed
tmohme opened this issue Nov 18, 2020 · 3 comments
Closed

NullPointerException in jqwik on test failure #132

tmohme opened this issue Nov 18, 2020 · 3 comments
Labels

Comments

@tmohme
Copy link
Contributor

tmohme commented Nov 18, 2020

Testing Problem

With one tests with generated nullable parameters, I get a NullPointerException in jqwik when the test fails.

This is the stack trace I get on the console:

java.lang.NullPointerException
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
	at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
	at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
	at net.jqwik.engine.execution.reporting.ParameterChangesDetector.atLeastOneChangedParameterHasEqualsImplementation(ParameterChangesDetector.java:19)
	at net.jqwik.engine.execution.reporting.ParameterChangesDetector.haveParametersChanged(ParameterChangesDetector.java:12)
	at net.jqwik.engine.execution.reporting.ExecutionResultReport.reportParameterChanges(ExecutionResultReport.java:91)
	at net.jqwik.engine.execution.reporting.ExecutionResultReport.lambda$appendSamples$0(ExecutionResultReport.java:65)
	at java.base/java.util.Optional.ifPresent(Optional.java:183)
	at net.jqwik.engine.execution.reporting.ExecutionResultReport.appendSamples(ExecutionResultReport.java:59)
	at net.jqwik.engine.execution.reporting.ExecutionResultReport.buildJqwikReport(ExecutionResultReport.java:49)
	at net.jqwik.engine.execution.reporting.ExecutionResultReport.from(ExecutionResultReport.java:33)
	at net.jqwik.engine.execution.PropertyMethodExecutor.reportResult(PropertyMethodExecutor.java:134)
	at net.jqwik.engine.execution.PropertyMethodExecutor.executePropertyMethod(PropertyMethodExecutor.java:99)
	at net.jqwik.engine.execution.PropertyMethodExecutor.execute(PropertyMethodExecutor.java:42)
	at net.jqwik.engine.execution.PropertyTaskCreator.executeTestMethod(PropertyTaskCreator.java:128)
	at net.jqwik.engine.execution.PropertyTaskCreator.lambda$createTask$1(PropertyTaskCreator.java:55)
	at net.jqwik.engine.execution.pipeline.ExecutionTask$1.lambda$execute$0(ExecutionTask.java:31)
	at net.jqwik.engine.execution.lifecycle.CurrentTestDescriptor.runWithDescriptor(CurrentTestDescriptor.java:27)
	at net.jqwik.engine.execution.pipeline.ExecutionTask$1.execute(ExecutionTask.java:31)
	at net.jqwik.engine.execution.pipeline.ExecutionPipeline.runToTermination(ExecutionPipeline.java:82)
	at net.jqwik.engine.execution.JqwikExecutor.execute(JqwikExecutor.java:46)
	at net.jqwik.engine.JqwikTestEngine.executeTests(JqwikTestEngine.java:77)
	at net.jqwik.engine.JqwikTestEngine.execute(JqwikTestEngine.java:61)

But when I stop the program when the NPE happens (via exception-breakpoint in IntelliJ), I find the following:

  • the NPE actually happens in net.jqwik.engine.execution.reporting.ArrayReportingFormat#appliesTo because the actual argument is null
  • some levels up in the call hierarchy, in net.jqwik.engine.execution.reporting.SampleReporter#reportParameters, the parameterName is set, while the parameterValue is null

This is the stack trace I get with the exception breakpoint:

appliesTo:10, ArrayReportingFormat (net.jqwik.engine.execution.reporting)
lambda$reportingFormatFinder$3:123, ValueReport (net.jqwik.engine.execution.reporting)
test:-1, 1064197277 (net.jqwik.engine.execution.reporting.ValueReport$$Lambda$509)
accept:176, ReferencePipeline$2$1 (java.util.stream)
tryAdvance:1632, ArrayList$ArrayListSpliterator (java.util)
forEachWithCancel:127, ReferencePipeline (java.util.stream)
copyIntoWithCancel:502, AbstractPipeline (java.util.stream)
copyInto:488, AbstractPipeline (java.util.stream)
wrapAndCopyInto:474, AbstractPipeline (java.util.stream)
evaluateSequential:150, FindOps$FindOp (java.util.stream)
evaluate:234, AbstractPipeline (java.util.stream)
findFirst:543, ReferencePipeline (java.util.stream)
lambda$reportingFormatFinder$4:128, ValueReport (net.jqwik.engine.execution.reporting)
find:-1, 285919215 (net.jqwik.engine.execution.reporting.ValueReport$$Lambda$508)
of:31, ValueReport (net.jqwik.engine.execution.reporting)
of:21, ValueReport (net.jqwik.engine.execution.reporting)
of:16, ValueReport (net.jqwik.engine.execution.reporting)
reportParameters:74, SampleReporter (net.jqwik.engine.execution.reporting)
reportTo:67, SampleReporter (net.jqwik.engine.execution.reporting)
buildReports:57, DefaultReporter (net.jqwik.engine.execution.reporting)
publishReports:50, DefaultReporter (net.jqwik.engine.execution.reporting)
lambda$createFalsifiedSampleReporter$0:202, GenericProperty (net.jqwik.engine.properties)
accept:-1, 734897812 (net.jqwik.engine.properties.GenericProperty$$Lambda$474)
lambda$shrink$1:59, PropertyShrinker (net.jqwik.engine.properties.shrinking)
accept:-1, 976807132 (net.jqwik.engine.properties.shrinking.PropertyShrinker$$Lambda$477)
shrink:72, AbstractSampleShrinker (net.jqwik.engine.properties.shrinking)
shrinkSingleParameter:43, OneAfterTheOtherParameterShrinker (net.jqwik.engine.properties.shrinking)
shrink:25, OneAfterTheOtherParameterShrinker (net.jqwik.engine.properties.shrinking)
shrinkOneParameterAfterTheOther:120, PropertyShrinker (net.jqwik.engine.properties.shrinking)
shrinkAsLongAsSampleImproves:107, PropertyShrinker (net.jqwik.engine.properties.shrinking)
lambda$shrink$3:77, PropertyShrinker (net.jqwik.engine.properties.shrinking)
get:-1, 1075284181 (net.jqwik.engine.properties.shrinking.PropertyShrinker$$Lambda$479)
run$$$capture:1700, CompletableFuture$AsyncSupply (java.util.concurrent)
run:-1, CompletableFuture$AsyncSupply (java.util.concurrent)
 - Async stack trace
<init>:1686, CompletableFuture$AsyncSupply (java.util.concurrent)
asyncSupplyStage:1714, CompletableFuture (java.util.concurrent)
supplyAsync:1915, CompletableFuture (java.util.concurrent)
withTimeout:88, PropertyShrinker (net.jqwik.engine.properties.shrinking)
shrink:81, PropertyShrinker (net.jqwik.engine.properties.shrinking)
shrink:68, PropertyShrinker (net.jqwik.engine.properties.shrinking)
shrink:194, GenericProperty (net.jqwik.engine.properties)
shrinkAndCreateCheckResult:166, GenericProperty (net.jqwik.engine.properties)
check:70, GenericProperty (net.jqwik.engine.properties)
check:58, CheckedProperty (net.jqwik.engine.execution)
executeProperty:124, PropertyMethodExecutor (net.jqwik.engine.execution)
executeMethod:108, PropertyMethodExecutor (net.jqwik.engine.execution)
lambda$executePropertyMethod$0:88, PropertyMethodExecutor (net.jqwik.engine.execution)
lambda$static$0:43, AroundPropertyHook (net.jqwik.api.lifecycle)
lambda$wrap$0:26, HookSupport (net.jqwik.engine.execution.lifecycle)
aroundProperty:70, PropertyDefaults$PropertyDefaultsHook (net.jqwik.api)
lambda$wrap$1:31, HookSupport (net.jqwik.engine.execution.lifecycle)
lambda$wrap$0:26, HookSupport (net.jqwik.engine.execution.lifecycle)
aroundProperty:57, PropertyLifecycleMethodsHook (net.jqwik.engine.hooks.lifecycle)
lambda$wrap$1:31, HookSupport (net.jqwik.engine.execution.lifecycle)
lambda$wrap$0:26, HookSupport (net.jqwik.engine.execution.lifecycle)
aroundProperty:32, StatisticsHook (net.jqwik.engine.hooks.statistics)
lambda$wrap$1:31, HookSupport (net.jqwik.engine.execution.lifecycle)
lambda$wrap$0:26, HookSupport (net.jqwik.engine.execution.lifecycle)
aroundProperty:13, AutoCloseableHook (net.jqwik.engine.hooks.lifecycle)
lambda$wrap$1:31, HookSupport (net.jqwik.engine.execution.lifecycle)
executePropertyMethod:86, PropertyMethodExecutor (net.jqwik.engine.execution)
execute:42, PropertyMethodExecutor (net.jqwik.engine.execution)
executeTestMethod:128, PropertyTaskCreator (net.jqwik.engine.execution)
lambda$createTask$1:55, PropertyTaskCreator (net.jqwik.engine.execution)
lambda$execute$0:31, ExecutionTask$1 (net.jqwik.engine.execution.pipeline)
runWithDescriptor:27, CurrentTestDescriptor (net.jqwik.engine.execution.lifecycle)
execute:31, ExecutionTask$1 (net.jqwik.engine.execution.pipeline)
runToTermination:82, ExecutionPipeline (net.jqwik.engine.execution.pipeline)
execute:46, JqwikExecutor (net.jqwik.engine.execution)
executeTests:77, JqwikTestEngine (net.jqwik.engine)
execute:61, JqwikTestEngine (net.jqwik.engine)
execute:108, EngineExecutionOrchestrator (org.junit.platform.launcher.core)
execute:88, EngineExecutionOrchestrator (org.junit.platform.launcher.core)
lambda$execute$0:54, EngineExecutionOrchestrator (org.junit.platform.launcher.core)
withInterceptedStreams:67, EngineExecutionOrchestrator (org.junit.platform.launcher.core)
execute:52, EngineExecutionOrchestrator (org.junit.platform.launcher.core)
execute:96, DefaultLauncher (org.junit.platform.launcher.core)
execute:75, DefaultLauncher (org.junit.platform.launcher.core)
startRunnerWithArgs:71, JUnit5IdeaTestRunner (com.intellij.junit5)
startRunnerWithArgs:33, IdeaTestRunner$Repeater (com.intellij.rt.junit)
prepareStreamsAndStart:220, JUnitStarter (com.intellij.rt.junit)
main:53, JUnitStarter (com.intellij.rt.junit)

Interestingly, adding a @Report(Reporting.FALSIFIED) annotation to the test produces the expected output and has no problem with the (test-) parameter in question being null.
All this happens with jqwik 1.3.8 and Java 11.

Suggested Solution

We should not get a NullPointerException.

Discussion

If you need more information (like a reproducer) please let me know.

@jlink
Copy link
Owner

jlink commented Nov 18, 2020

Thanks for the catch. I’ll have a look at it.

jlink added a commit that referenced this issue Nov 19, 2020
@jlink
Copy link
Owner

jlink commented Nov 19, 2020

Fixed in 7fb5b64

Available in 1.3.9-SNAPSHOT

@jlink jlink closed this as completed Nov 19, 2020
@jlink jlink removed the in progress label Nov 19, 2020
@tmohme
Copy link
Contributor Author

tmohme commented Nov 19, 2020

Fyi: Just want to confirm that 1.3.9-SNAPSHOT solves my problem.
Thank you for the super fast reaction!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants