Inappropriate blocking method call for serialization (2)

[cj848]

https://github.com/cj848/blockhound-kotlin-jackson-module

I found an inappropriate blocking call during testing using blockhound.
More details are in the github source I attached.
If you remove the dependency of jackson-module-kotlin from the source I attached, the test of the code passes.

If you uncomment line 12 of the LoggingBlockHoundIntegration class
The exact cause below appears.

I am very uncomfortable with this situation. Because I am using data class very much. please fix this issue. Thank you for always.

=========================================================
java.lang.Exception: java.io.RandomAccessFile#readBytes
at kr.co.kcd.example.LoggingBlockHoundIntegration$applyTo$1.accept(LoggingBlockHoundIntegration.kt:11)
at kr.co.kcd.example.LoggingBlockHoundIntegration$applyTo$1.accept(LoggingBlockHoundIntegration.kt:8)
at reactor.blockhound.BlockHound$Builder.lambda$install$8(BlockHound.java:383)
at reactor.blockhound.BlockHoundRuntime.checkBlocking(BlockHoundRuntime.java:89)
at java.base/java.io.RandomAccessFile.readBytes(RandomAccessFile.java)
at java.base/java.io.RandomAccessFile.read(RandomAccessFile.java:408)
at java.base/java.util.zip.ZipFile$Source.readAt(ZipFile.java:1252)
at java.base/java.util.zip.ZipFile$ZipFileInputStream.read(ZipFile.java:958)
at java.base/java.util.zip.ZipFile$ZipFileInflaterInputStream.fill(ZipFile.java:460)
at java.base/java.util.zip.InflaterInputStream.read(InflaterInputStream.java:159)
at java.base/java.util.zip.InflaterInputStream.read(InflaterInputStream.java:123)
at java.base/java.io.FilterInputStream.read(FilterInputStream.java:83)
at java.base/java.io.DataInputStream.readInt(DataInputStream.java:392)
at kotlin.reflect.jvm.internal.impl.metadata.builtins.BuiltInsBinaryVersion$Companion.readFrom(BuiltInsBinaryVersion.kt:29)
at kotlin.reflect.jvm.internal.impl.serialization.deserialization.builtins.BuiltInsPackageFragmentImpl$Companion.create(BuiltInsPackageFragmentImpl.kt:38)
at kotlin.reflect.jvm.internal.impl.builtins.jvm.JvmBuiltInsPackageFragmentProvider.findPackage(JvmBuiltInsPackageFragmentProvider.kt:58)
at kotlin.reflect.jvm.internal.impl.serialization.deserialization.AbstractDeserializedPackageFragmentProvider$fragments$1.invoke(AbstractDeserializedPackageFragmentProvider.kt:34)
at kotlin.reflect.jvm.internal.impl.serialization.deserialization.AbstractDeserializedPackageFragmentProvider$fragments$1.invoke(AbstractDeserializedPackageFragmentProvider.kt:26)
at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$MapBasedMemoizedFunction.invoke(LockBasedStorageManager.java:512)
at kotlin.reflect.jvm.internal.impl.serialization.deserialization.AbstractDeserializedPackageFragmentProvider.getPackageFragments(AbstractDeserializedPackageFragmentProvider.kt:41)
at kotlin.reflect.jvm.internal.impl.descriptors.impl.CompositePackageFragmentProvider.getPackageFragments(CompositePackageFragmentProvider.kt:31)
at kotlin.reflect.jvm.internal.impl.descriptors.impl.CompositePackageFragmentProvider.getPackageFragments(CompositePackageFragmentProvider.kt:31)
at kotlin.reflect.jvm.internal.impl.descriptors.impl.LazyPackageViewDescriptorImpl$fragments$2.invoke(LazyPackageViewDescriptorImpl.kt:37)
at kotlin.reflect.jvm.internal.impl.descriptors.impl.LazyPackageViewDescriptorImpl$fragments$2.invoke(LazyPackageViewDescriptorImpl.kt:30)
at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$LockBasedLazyValue.invoke(LockBasedStorageManager.java:355)
at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$LockBasedNotNullLazyValue.invoke(LockBasedStorageManager.java:474)
at kotlin.reflect.jvm.internal.impl.storage.StorageKt.getValue(storage.kt:42)
at kotlin.reflect.jvm.internal.impl.descriptors.impl.LazyPackageViewDescriptorImpl.getFragments(LazyPackageViewDescriptorImpl.kt)
at kotlin.reflect.jvm.internal.impl.descriptors.impl.LazyPackageViewDescriptorImpl$memberScope$1.invoke(LazyPackageViewDescriptorImpl.kt:41)
at kotlin.reflect.jvm.internal.impl.descriptors.impl.LazyPackageViewDescriptorImpl$memberScope$1.invoke(LazyPackageViewDescriptorImpl.kt:30)
at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$LockBasedLazyValue.invoke(LockBasedStorageManager.java:355)
at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$LockBasedNotNullLazyValue.invoke(LockBasedStorageManager.java:474)
at kotlin.reflect.jvm.internal.impl.resolve.scopes.LazyScopeAdapter.getWorkerScope(LazyScopeAdapter.kt:23)
at kotlin.reflect.jvm.internal.impl.resolve.scopes.AbstractScopeAdapter.getContributedClassifier(AbstractScopeAdapter.kt:44)
at kotlin.reflect.jvm.internal.impl.descriptors.FindClassInModuleKt.findClassifierAcrossModuleDependencies(findClassInModule.kt:25)
at kotlin.reflect.jvm.internal.impl.descriptors.FindClassInModuleKt.findClassAcrossModuleDependencies(findClassInModule.kt:40)
at kotlin.reflect.jvm.internal.impl.serialization.deserialization.TypeDeserializer.computeClassDescriptor(TypeDeserializer.kt:221)
at kotlin.reflect.jvm.internal.impl.serialization.deserialization.TypeDeserializer.access$computeClassDescriptor(TypeDeserializer.kt:23)
at kotlin.reflect.jvm.internal.impl.serialization.deserialization.TypeDeserializer$classDescriptors$1.invoke(TypeDeserializer.kt:32)
at kotlin.reflect.jvm.internal.impl.serialization.deserialization.TypeDeserializer$classDescriptors$1.invoke(TypeDeserializer.kt:23)
at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$MapBasedMemoizedFunction.invoke(LockBasedStorageManager.java:512)
at kotlin.reflect.jvm.internal.impl.serialization.deserialization.TypeDeserializer.typeConstructor(TypeDeserializer.kt:113)
at kotlin.reflect.jvm.internal.impl.serialization.deserialization.TypeDeserializer.simpleType(TypeDeserializer.kt:75)
at kotlin.reflect.jvm.internal.impl.serialization.deserialization.TypeDeserializer.type(TypeDeserializer.kt:63)
at kotlin.reflect.jvm.internal.impl.serialization.deserialization.MemberDeserializer.valueParameters(MemberDeserializer.kt:417)
at kotlin.reflect.jvm.internal.impl.serialization.deserialization.MemberDeserializer.loadConstructor(MemberDeserializer.kt:342)
at kotlin.reflect.jvm.internal.impl.serialization.deserialization.descriptors.DeserializedClassDescriptor.computePrimaryConstructor(DeserializedClassDescriptor.kt:122)
at kotlin.reflect.jvm.internal.impl.serialization.deserialization.descriptors.DeserializedClassDescriptor.access$computePrimaryConstructor(DeserializedClassDescriptor.kt:34)
at kotlin.reflect.jvm.internal.impl.serialization.deserialization.descriptors.DeserializedClassDescriptor$primaryConstructor$1.invoke(DeserializedClassDescriptor.kt:65)
at kotlin.reflect.jvm.internal.impl.serialization.deserialization.descriptors.DeserializedClassDescriptor$primaryConstructor$1.invoke(DeserializedClassDescriptor.kt:34)
at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$LockBasedLazyValue.invoke(LockBasedStorageManager.java:355)
at kotlin.reflect.jvm.internal.impl.serialization.deserialization.descriptors.DeserializedClassDescriptor.getUnsubstitutedPrimaryConstructor(DeserializedClassDescriptor.kt:126)
at kotlin.reflect.jvm.internal.impl.serialization.deserialization.descriptors.DeserializedClassDescriptor.computeConstructors(DeserializedClassDescriptor.kt:129)
at kotlin.reflect.jvm.internal.impl.serialization.deserialization.descriptors.DeserializedClassDescriptor.access$computeConstructors(DeserializedClassDescriptor.kt:34)
at kotlin.reflect.jvm.internal.impl.serialization.deserialization.descriptors.DeserializedClassDescriptor$constructors$1.invoke(DeserializedClassDescriptor.kt:66)
at kotlin.reflect.jvm.internal.impl.serialization.deserialization.descriptors.DeserializedClassDescriptor$constructors$1.invoke(DeserializedClassDescriptor.kt:34)
at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$LockBasedLazyValue.invoke(LockBasedStorageManager.java:355)
at kotlin.reflect.jvm.internal.impl.storage.LockBasedStorageManager$LockBasedNotNullLazyValue.invoke(LockBasedStorageManager.java:474)
at kotlin.reflect.jvm.internal.impl.serialization.deserialization.descriptors.DeserializedClassDescriptor.getConstructors(DeserializedClassDescriptor.kt:137)
at kotlin.reflect.jvm.internal.KClassImpl.getConstructorDescriptors(KClassImpl.kt:200)
at kotlin.reflect.jvm.internal.KClassImpl$Data$constructors$2.invoke(KClassImpl.kt:91)
at kotlin.reflect.jvm.internal.KClassImpl$Data$constructors$2.invoke(KClassImpl.kt:44)
at kotlin.reflect.jvm.internal.ReflectProperties$LazySoftVal.invoke(ReflectProperties.java:92)
at kotlin.reflect.jvm.internal.ReflectProperties$Val.getValue(ReflectProperties.java:31)
at kotlin.reflect.jvm.internal.KClassImpl$Data.getConstructors(KClassImpl.kt)
at kotlin.reflect.jvm.internal.KClassImpl.getConstructors(KClassImpl.kt:235)
at kotlin.reflect.jvm.ReflectJvmMapping.getKotlinFunction(ReflectJvmMapping.kt:144)
at com.fasterxml.jackson.module.kotlin.KotlinNamesAnnotationIntrospector.findKotlinParameterName(KotlinNamesAnnotationIntrospector.kt:115)
at com.fasterxml.jackson.module.kotlin.KotlinNamesAnnotationIntrospector.findImplicitPropertyName(KotlinNamesAnnotationIntrospector.kt:31)
at com.fasterxml.jackson.databind.introspect.AnnotationIntrospectorPair.findImplicitPropertyName(AnnotationIntrospectorPair.java:490)
at com.fasterxml.jackson.databind.introspect.POJOPropertiesCollector._addCreatorParam(POJOPropertiesCollector.java:485)
at com.fasterxml.jackson.databind.introspect.POJOPropertiesCollector._addCreators(POJOPropertiesCollector.java:465)
at com.fasterxml.jackson.databind.introspect.POJOPropertiesCollector.collectAll(POJOPropertiesCollector.java:313)
at com.fasterxml.jackson.databind.introspect.POJOPropertiesCollector.getJsonValueAccessor(POJOPropertiesCollector.java:196)
at com.fasterxml.jackson.databind.introspect.BasicBeanDescription.findJsonValueAccessor(BasicBeanDescription.java:252)
at com.fasterxml.jackson.databind.ser.BasicSerializerFactory.findSerializerByAnnotations(BasicSerializerFactory.java:346)
at com.fasterxml.jackson.databind.ser.BeanSerializerFactory._createSerializer2(BeanSerializerFactory.java:216)
at com.fasterxml.jackson.databind.ser.BeanSerializerFactory.createSerializer(BeanSerializerFactory.java:165)
at com.fasterxml.jackson.databind.SerializerProvider._createUntypedSerializer(SerializerProvider.java:1388)
at com.fasterxml.jackson.databind.SerializerProvider._createAndCacheUntypedSerializer(SerializerProvider.java:1336)
at com.fasterxml.jackson.databind.SerializerProvider._findExplicitUntypedSerializer(SerializerProvider.java:1305)
at com.fasterxml.jackson.databind.ser.DefaultSerializerProvider.hasSerializerFor(DefaultSerializerProvider.java:260)
at com.fasterxml.jackson.databind.ObjectMapper.canSerialize(ObjectMapper.java:3005)
at org.springframework.http.codec.json.AbstractJackson2Encoder.canEncode(AbstractJackson2Encoder.java:107)
at org.springframework.http.codec.EncoderHttpMessageWriter.canWrite(EncoderHttpMessageWriter.java:107)
at org.springframework.web.reactive.function.BodyInserters.lambda$writeWithMessageWriters$9(BodyInserters.java:377)
at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:176)
at java.base/java.util.ArrayList$ArrayListSpliterator.tryAdvance(ArrayList.java:1598)
at java.base/java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:127)
at java.base/java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:502)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:488)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at java.base/java.util.stream.FindOps$FindOp.evaluateSequential(FindOps.java:150)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.findFirst(ReferencePipeline.java:543)
at org.springframework.web.reactive.function.BodyInserters.writeWithMessageWriters(BodyInserters.java:378)
at org.springframework.web.reactive.function.BodyInserters.lambda$fromValue$1(BodyInserters.java:98)
at org.springframework.web.reactive.function.client.DefaultClientRequestBuilder$BodyInserterRequest.writeTo(DefaultClientRequestBuilder.java:249)
at org.springframework.web.reactive.function.client.ExchangeFunctions$DefaultExchangeFunction.lambda$exchange$1(ExchangeFunctions.java:104)
at org.springframework.http.client.reactive.ReactorClientHttpConnector.lambda$connect$2(ReactorClientHttpConnector.java:110)
at reactor.netty.http.client.HttpClientConnect$HttpClientHandler.requestWithBody(HttpClientConnect.java:607)
at reactor.netty.http.client.HttpClientConnect$HttpIOHandlerObserver.lambda$onStateChange$0(HttpClientConnect.java:434)
at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:44)
at reactor.netty.http.client.HttpClientConnect$HttpIOHandlerObserver.onStateChange(HttpClientConnect.java:435)
at reactor.netty.ReactorNetty$CompositeConnectionObserver.onStateChange(ReactorNetty.java:514)
at reactor.netty.resources.PooledConnectionProvider$DisposableAcquire.onStateChange(PooledConnectionProvider.java:523)
at reactor.netty.resources.PooledConnectionProvider$PooledConnection.onStateChange(PooledConnectionProvider.java:429)
at reactor.netty.channel.ChannelOperationsHandler.channelActive(ChannelOperationsHandler.java:60)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:230)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:216)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelActive(AbstractChannelHandlerContext.java:209)
at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelActive(CombinedChannelDuplexHandler.java:412)
at io.netty.channel.ChannelInboundHandlerAdapter.channelActive(ChannelInboundHandlerAdapter.java:69)
at io.netty.channel.CombinedChannelDuplexHandler.channelActive(CombinedChannelDuplexHandler.java:211)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:230)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:216)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelActive(AbstractChannelHandlerContext.java:209)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelActive(DefaultChannelPipeline.java:1398)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:230)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:216)
at io.netty.channel.DefaultChannelPipeline.fireChannelActive(DefaultChannelPipeline.java:895)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:305)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:335)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:702)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:830)

[cj848]

If I apply below codes to the attached github, the exception disappears.
But I don't know if this is the right solution.
is this right solution??

in LoggingBlockHoundIntegration

builder
.allowBlockingCallsInside(ObjectReader::class.qualifiedName, "readValue")
.allowBlockingCallsInside(ObjectMapper::class.qualifiedName, "canSerialize")

[62mkv]

does that happen with pure Java version as well?

[62mkv]

If I apply below codes to the attached github, the exception disappears.
But I don't know if this is the right solution.
is this right solution??

in LoggingBlockHoundIntegration

builder
.allowBlockingCallsInside(ObjectReader::class.qualifiedName, "readValue")
.allowBlockingCallsInside(ObjectMapper::class.qualifiedName, "canSerialize")

this is the right solution if you're certain that blocking calls from those places is fine for your application. In general, I wouldn't assume that.. but upon closer inspection it might end up being "fine(-ish)"

[cj848]

does that happen with pure Java version as well?

Removing dependency jackson-module-kotlin is equivalent to using Pure java.
So, my answer is yes.

If I apply below codes to the attached github, the exception disappears.
But I don't know if this is the right solution.
is this right solution??
in LoggingBlockHoundIntegration
builder
.allowBlockingCallsInside(ObjectReader::class.qualifiedName, "readValue")
.allowBlockingCallsInside(ObjectMapper::class.qualifiedName, "canSerialize")

this is the right solution if you're certain that blocking calls from those places is fine for your application. In general, I wouldn't assume that.. but upon closer inspection it might end up being "fine(-ish)"

In my opinion, that codes in jackson-module-kotlin will not affect performance unless it is external I/O.
Therefore, BlockHound provides a way to handle exceptions as above. For reference, BlockHoundIntegration is implemented in netty or spring, and only the blocking methods that do not affect the known performance are handled.

My question is, how does this module also provide an implementation of such a BlockHoundIntegration?

[62mkv]

Removing dependency jackson-module-kotlin is equivalent to using Pure java. So, my answer is yes.

why is this issue in this repository, then?

[cj848]

Removing dependency jackson-module-kotlin is equivalent to using Pure java. So, my answer is yes.

why is this issue in this repository, then?

I think I misunderstood the question. When dependency is removed, pure java version of jackson works and blockhound does not detect blocking.
However, blocking is detected by adding the jackson-module-kotlin dependency.

[cj848]

So eventually blocking occurs after adding this library dependency.
The question I originally intended is because there are no points in the existing jackson library that can cause blocking, but only in this library. How about adding it for Reactive users?

[62mkv]

as @bsideup highlighted in the chat:

I don't think the issue belongs to Jackson, unless they claim that it is non-blocking. From the stacktrace, it looks like Kotlin is doing some IO from the kotlin.reflect.jvm module. you may whitelist the Kotlin reflect call if you feel okay with it

[cj848]

@62mkv Ok. I see. i just proposal for this repository owners.
i have been applied already.

[cowtowncoder]

While Jackson streaming parser (JsonParser) of some backends (namely, JSON, Smile) does implement "non-blocking" (aka "async") decoding mode, regular API for streaming as well as whole databind builds on standard JDK i/o classes, which indicate blocking.

When using non-streaming input (String, byte[], ByteBuffer) input abstractions are still used although actual operation can not block.
This is what Jackson guarantees.

I am not familiar with "blockhound" (I assume it is this? https://github.com/reactor/BlockHound -- in future, please add a link or brief description instead of assuming everyone knows the tool in question), but from quick reading I am not sure there is anything Jackson could or should do.

Also: as a user, if you want non-blocking handling, it is not a good idea to simply assume libraries do that -- good to first investigate and only then use.

[cj848]

@cowtowncoder
I'm sorry I didn't explain it.
Rather than wishing the library to do everything, I was talking about examples from spring or netty.
See the link below.

https://github.com/spring-projects/spring-framework/blob/2ae91404d1424bfc16b2dca62e4c6f8f3e9174f6/spring-core/src/main/java/org/springframework/core/ReactiveAdapterRegistry.java#L371
it is a spring framework 'core' library and that library have BlockHoundIntegration Implementation.

https://github.com/netty/netty/blob/fcf55fcf718af1e4cae85eb86cdb679094186aed/common/src/main/java/io/netty/util/internal/Hidden.java#L39
and also Netty too.

i just proposal for this repository owner. so I'm sorry if you feel bad.

[cowtowncoder]

@cj848 That is fine, don't worry. This is just something I was not familiar with and hoped to explain how Jackson handles blocking/non-blocking i/o, if that helps. But it sounds like this is related to integration by Spring, which may be indicating certain behavior -- I know some Spring components actually use Jackson's non-blocking parsing; and other parts probably know well which input source they use (like byte[] instead of InputStream).

[cj848]

@cowtowncoder
To explain that, there seems to be blocking code in the reflection of kotlin as bside mentioned above.
And this doesn't seem to be the fault of this library. However, the first question was whether it could be solved because it seems to be a problem when using this library.
When I first asked, I didn't know about that solution, and after more searching, I found that the solution was to register an exception.
After that, I was asking if it could be solved. The background is the same as described above.

To return the question, I wonder if there are any modifiable parts of this module other than registering kotlin reflection as an exception. If not, it would be nice to just register as an exception.

[libinglong]

i dont have any idea to handle it.Finally I just suppress it

[dinomite]

Closing in favor of #313