Kotlin `object` deserialization with @JsonCreator fails in Jackson 2.18.0

[silmeth]

Search before asking

• I searched in the issues and found nothing similar.
• I searched in the issues of databind and other modules used and found nothing similar.
• I have confirmed that the problem only occurs when using Kotlin.

Describe the bug

In Jackson 2.17 and earlier, object which implements a @JsonCreator @JvmStatic fun deserialize() function serializes and deserializes successfully. After upgrading to Jackson 2.18.0 it fails.

To Reproduce

package jackson.test

import com.fasterxml.jackson.annotation.JsonCreator
import com.fasterxml.jackson.annotation.JsonInclude
import com.fasterxml.jackson.databind.ObjectMapper
import com.fasterxml.jackson.datatype.jdk8.Jdk8Module
import com.fasterxml.jackson.module.kotlin.readValue
import com.fasterxml.jackson.module.kotlin.registerKotlinModule
import org.junit.jupiter.api.Assertions.assertEquals
import org.junit.jupiter.api.Test

object Foo {
    override fun toString(): String = "Foo()"

    @JvmStatic
    @JsonCreator
    fun deserialize(): Foo {
        println("Foo.deserialize() called")
        return Foo
    }
}

class JacksonTest {
    private val mapper =
        ObjectMapper()
            .copy()
            .setSerializationInclusion(JsonInclude.Include.NON_ABSENT)
            .registerModule(Jdk8Module())
            .registerKotlinModule()

    @Test
    fun shouldDeserializeSimpleObject() {
        val value = Foo
        val serialized = mapper.writeValueAsString(value)
        println(serialized)
        val deserialized = mapper.readValue<Foo>(serialized)

        assertEquals(value, deserialized)
    }
}

Expected behavior

Test passes with stdout (that’s the behaviour with Jackson 2.17.0):

{}
Foo.deserialize() called

Instead what happens with 2.18.0 is the test fails with:

Test output

com.fasterxml.jackson.databind.exc.ValueInstantiationException: Cannot construct instance of `jackson.test.Foo`, problem: `java.lang.NullPointerException`
 at [Source: REDACTED (`StreamReadFeature.INCLUDE_SOURCE_IN_LOCATION` disabled); line: 1, column: 2]
	at app//com.fasterxml.jackson.databind.exc.ValueInstantiationException.from(ValueInstantiationException.java:47)
	at app//com.fasterxml.jackson.databind.DeserializationContext.instantiationException(DeserializationContext.java:2015)
	at app//com.fasterxml.jackson.databind.DeserializationContext.handleInstantiationProblem(DeserializationContext.java:1426)
	at app//com.fasterxml.jackson.databind.deser.BeanDeserializerBase.wrapInstantiationProblem(BeanDeserializerBase.java:2010)
	at app//com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:536)
	at app//com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1497)
	at app//com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:348)
	at app//com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:185)
	at app//com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:342)
	at app//com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4917)
	at app//com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3860)
	at app//com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3843)
	at app//jackson.test.JacksonTest.shouldDeserializeSimpleObject(JacksonTest.kt:45)
	at java.base@17.0.12/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base@17.0.12/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base@17.0.12/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base@17.0.12/java.lang.reflect.Method.invoke(Method.java:569)
	at app//org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:766)
	at app//org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
	at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
	at app//org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156)
	at app//org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147)
	at app//org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:86)
	at app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
	at app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
	at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
	at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
	at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
	at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
	at app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92)
	at app//org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86)
	at app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$8(TestMethodTestDescriptor.java:217)
	at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:213)
	at app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:138)
	at app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:68)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:156)
	at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:146)
	at app//org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:144)
	at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:143)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:100)
	at java.base@17.0.12/java.util.ArrayList.forEach(ArrayList.java:1511)
	at app//org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:160)
	at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:146)
	at app//org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:144)
	at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:143)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:100)
	at java.base@17.0.12/java.util.ArrayList.forEach(ArrayList.java:1511)
	at app//org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:160)
	at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:146)
	at app//org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:144)
	at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:143)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:100)
	at app//org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35)
	at app//org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
	at app//org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54)
	at app//org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:107)
	at app//org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
	at app//org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
	at app//org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
	at app//org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
	at app//org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114)
	at app//org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86)
	at app//org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:124)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.access$000(JUnitPlatformTestClassProcessor.java:99)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor.stop(JUnitPlatformTestClassProcessor.java:94)
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:63)
	at java.base@17.0.12/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base@17.0.12/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base@17.0.12/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base@17.0.12/java.lang.reflect.Method.invoke(Method.java:569)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:92)
	at jdk.proxy1/jdk.proxy1.$Proxy20.stop(Unknown Source)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker$3.run(TestWorker.java:200)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:132)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:103)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:63)
	at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:121)
	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:71)
	at app//worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
	at app//worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)
Caused by: java.lang.NullPointerException
	at com.fasterxml.jackson.databind.util.internal.PrivateMaxEntriesMap.checkNotNull(PrivateMaxEntriesMap.java:226)
	at com.fasterxml.jackson.databind.util.internal.PrivateMaxEntriesMap.put(PrivateMaxEntriesMap.java:648)
	at com.fasterxml.jackson.databind.util.internal.PrivateMaxEntriesMap.putIfAbsent(PrivateMaxEntriesMap.java:633)
	at com.fasterxml.jackson.databind.util.LRUMap.putIfAbsent(LRUMap.java:64)
	at com.fasterxml.jackson.module.kotlin.ReflectionCache.valueCreatorFromJava(ReflectionCache.kt:96)
	at com.fasterxml.jackson.module.kotlin.KotlinValueInstantiator.createFromObjectWith(KotlinValueInstantiator.kt:52)
	at com.fasterxml.jackson.databind.deser.impl.PropertyBasedCreator.build(PropertyBasedCreator.java:214)
	at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:534)
	... 92 more

and stdout – showing that it got correctly serialized to empty object, but @JsonCreator never gets called:

{}

Versions

Kotlin: 2.0.21
Jackson-module-kotlin: 2.18.0
Jackson-databind: 2.18.0

Additional context

Perhaps related to FasterXML/jackson-databind#4742 (but here it’s a simple object in Kotlin instead).

[k163377]

@cowtowncoder
In 2.18, have you made any changes to the caller of StdValueInstantiator.configureFromObjectSettings, in particular withArgsCreator?
I have investigated the issue reported here, and it seems that for no-argument factories, StdValueInstantiator._withArgsCreator was null until 2.17, whereas in 2.18 a non-null value is passed. It seems to be 2.18.
I would appreciate any help as it has been difficult to determine the cause.

[cowtowncoder]

@k163377 Although number of changes for 2.18 is big enough that I don't have full recollection, but I don't think I made any changes targeting that method.
It is possible something with POJO Property Introspection could have necessitated changes tho. Git blame would show who changed code last.

But @JooHyukKim did work on better support for any setters and that could have resulted in changes in this class

One more question -- is there any change between 2.18.0 and current 2.18 (2.18.0-SNAPSHOT)?

One last thing: this late-stage change for 2.18.0 could be relevant here:

FasterXML/jackson-databind#4688

(via PR FasterXML/jackson-databind#4689 )

[JooHyukKim]

FYI, the mentioned any-setter job is as follows

• jackson-databind#4558 or
• jackson-databind#4720

Also, I will also try to look into the solution, to figure out solution. Won't be so soon tho.

[cowtowncoder]

For testing, what may help slightly is the release of 2.18.1 (happening now), can avoid use of SNAPSHOT versions (except for fixes that go in 2.18 branch from now on etc).

[JooHyukKim]

First Observation

First, this is where call paths diverges (between 2.17 and 2.18).
The crossroad is at _vanillaProcessing check in BeanDeserializer.deserialize().
In versions...

• 2.17, _vanillaProcessing is true so invokes vanillaDeserialize(), where as
• 2.18, _vanillaProcessing is false then proceeds to deserializeFromObject(p, ctxt)

Below is the method in question.

Second Observation

Another thing is that hitting ...

((KotlinValueInstantiator) BeanDeserializer._valueInstantiator)._defaultCreator

... on IntelliJ debugger returns in versions...

• 2.17, the creator method is returns, where as
• 2.18, just plain constructor is returned.

Assumptions

Looking at _vanillaProcessing changed value, seems like object class with creator is was not accounted for moving on to 2.18. Next step may be investigating _vanillaProcessing change behavior.

WDYT?

[k163377]

Issue submitted to databind with reproduction code.
FasterXML/jackson-databind#4777

This issue needs to be fixed in databind, but I will leave this issue open as I intend to add tests in kotlin-module after the fix.

[cowtowncoder]

Hmmmh. I am not yet fully convinced that 2.17 behavior is more correct than 2.18.
It comes down to semantics of what "default creator" is: should it

a. Represent either 0-argument Constructor or 0-argument Factory method (2.17)
b. Only represent 0-argument constructor; 0-argument Factory method handled as a special Properties-based Creator

If we can revert/change handling to (a) without breaking any tests, that would seem preferable.

Now: why does this change break Kotlin module? Is it calling 0-args constructor somehow, instead of factory method?
Original test is bit problematic as it uses mapper.writeValueAsString() so JSON used is not 100% clear.

I'll have a look at the new jackson-databind test PR, that may make more sense.

[k163377]

Now: why does this change break Kotlin module?

This is because KotlinValueInstantiator.createFromObjectWith is called as a result of the processing path change from 2.17.
https://github.com/FasterXML/jackson-module-kotlin/blob/2.19/src/main/kotlin/com/fasterxml/jackson/module/kotlin/KotlinValueInstantiator.kt#L47

The location where this path change is occurring is explained in @JooHyukKim 's comment.
#841 (comment)

[cowtowncoder]

@k163377 Ok; conceptually that call should be legal (as in, handling it like arguments-taking Creators should work the same way), but I agree it is at least sub-optimal: 0-args factory method cannot take any arguments so there is no need to go through the usual complications of buffering etc.

So I think ideally we'd figure out how to keep 0-args factory method as special case that does not go through regular properties-based-creator handling.

If that is not possible, Kotlin module should probably be changed to handle this case (whether it considers 0-args special or not).