rest service call with ssl blocks reactive thread

[urferr]

when using Spring webflux with reactor-netty i get a blocking exception when calling another rest service using ssl. Adding allowBlockingCallsInside("io.netty.handler.ssl.SslHandler", "handshake") to the Blockhound builder can be used as workaround

Stack trace:
at reactor.blockhound.BlockHound$Builder.lambda$new$0(BlockHound.java:196)
at reactor.blockhound.BlockHound$Builder.lambda$install$6(BlockHound.java:318)
at reactor.blockhound.BlockHoundRuntime.checkBlocking(BlockHoundRuntime.java:46)
at java.base/java.io.FileInputStream.readBytes(FileInputStream.java)
at java.base/java.io.FileInputStream.read(FileInputStream.java:279)
at java.base/java.io.FilterInputStream.read(FilterInputStream.java:133)
at java.base/sun.security.provider.NativePRNG$RandomIO.readFully(NativePRNG.java:424)
at java.base/sun.security.provider.NativePRNG$RandomIO.ensureBufferValid(NativePRNG.java:526)
at java.base/sun.security.provider.NativePRNG$RandomIO.implNextBytes(NativePRNG.java:545)
at java.base/sun.security.provider.NativePRNG.engineNextBytes(NativePRNG.java:220)
at java.base/java.security.SecureRandom.nextBytes(SecureRandom.java:741)
at java.base/sun.security.ssl.RandomCookie.(RandomCookie.java:67)
at java.base/sun.security.ssl.ClientHello$ClientHelloMessage.(ClientHello.java:93)
at java.base/sun.security.ssl.ClientHello$ClientHelloKickstartProducer.produce(ClientHello.java:639)
at java.base/sun.security.ssl.SSLHandshake.kickstart(SSLHandshake.java:515)
at java.base/sun.security.ssl.ClientHandshakeContext.kickstart(ClientHandshakeContext.java:104)
at java.base/sun.security.ssl.TransportContext.kickstart(TransportContext.java:228)
at java.base/sun.security.ssl.SSLEngineImpl.beginHandshake(SSLEngineImpl.java:103)
at io.netty.handler.ssl.SslHandler.handshake(SslHandler.java:1987)
at io.netty.handler.ssl.SslHandler.startHandshakeProcessing(SslHandler.java:1906)
at io.netty.handler.ssl.SslHandler.channelActive(SslHandler.java:2042)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:225)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:211)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelActive(AbstractChannelHandlerContext.java:204)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelActive(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:225)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelActive(AbstractChannelHandlerContext.java:211)
at io.netty.channel.DefaultChannelPipeline.fireChannelActive(DefaultChannelPipeline.java:907)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:305)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:335)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:688)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:635)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:552)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:514)
at io.netty.util.concurrent.SingleThreadEventExecutor$6.run(SingleThreadEventExecutor.java:1050)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:834)

• Reactor version(s) used: 3.3.0, reactor-netty 0.9.1
• Other relevant libraries versions: netty 4.1.43, spring 5.2.1
• JVM version (javar -version): openjdk version "11.0.2" 2018-10-16
• OS and version (eg uname -a): Linux pdvmdev15.profidatagroup.com 3.10.0-957.10.1.el7.x86_64 MonoHttpClientChannel subscribes to Mono with a null subscriber #1 SMP Thu Feb 7 07:12:53 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

[violetagg]

@urferr I would recommend you to report this to Netty project

[violetagg]

I'm closing this one. If it appears a Reactor Netty issue we can reopen it.

[rstoyanchev]

Link to Netty issue netty/netty#9834.

[violetagg]

Reopening in order to evaluate the suggested solution in netty/netty#9834

[violetagg]

need more clarification here netty/netty#9834 (comment)

[violetagg]

@urferr Fix will be available in Netty 4.1.46.Final netty/netty#9969
It would be great if you can test 4.1.46.Final-SNAPSHOT

[urferr]

Unfortunately i am not able to reproduce the original problem. Maybe its because some thirdparty library versions have slightly changed in the meantime:

• Reactor version(s) used: 3.3.1, reactor-netty 0.9.2
• Other relevant libraries versions: netty 4.1.43, spring 5.2.2
• JVM version (java -version): openjdk version "13.0.2"

[violetagg]

Netty updated to 4.1.46.Final 5fde036

[estigma88]

Hi all, I am getting a similar issue over java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted. I am using Netty 4.1.49.Final

Caused by: reactor.blockhound.BlockingOperationError: Blocking call! java.io.FileInputStream#readBytes
        at java.base/java.io.FileInputStream.readBytes(FileInputStream.java)
        at java.base/java.io.FileInputStream.read(FileInputStream.java:257)
        at java.base/java.util.Properties$LineReader.readLine(Properties.java:498)
        at java.base/java.util.Properties.load0(Properties.java:416)
        at java.base/java.util.Properties.load(Properties.java:404)
        at java.base/sun.security.util.UntrustedCertificates$1.run(UntrustedCertificates.java:60)
        at java.base/sun.security.util.UntrustedCertificates$1.run(UntrustedCertificates.java:54)
        at java.base/java.security.AccessController.doPrivileged(Native Method)
        at java.base/sun.security.util.UntrustedCertificates.<clinit>(UntrustedCertificates.java:54)
        at java.base/sun.security.provider.certpath.UntrustedChecker.check(UntrustedChecker.java:78)
        at java.base/java.security.cert.PKIXCertPathChecker.check(PKIXCertPathChecker.java:176)
        at java.base/sun.security.provider.certpath.PKIXCertPathValidator.validate(PKIXCertPathValidator.java:167)
        at java.base/sun.security.provider.certpath.PKIXCertPathValidator.validate(PKIXCertPathValidator.java:141)
        at java.base/sun.security.provider.certpath.PKIXCertPathValidator.engineValidate(PKIXCertPathValidator.java:80)
        at java.base/java.security.cert.CertPathValidator.validate(CertPathValidator.java:309)
        at java.base/sun.security.validator.PKIXValidator.doValidate(PKIXValidator.java:345)
        at java.base/sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:248)
        at java.base/sun.security.validator.Validator.validate(Validator.java:264)
        at java.base/sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:321)
        at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:279)
        at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:141)
        at io.netty.handler.ssl.ReferenceCountedOpenSslClientContext$ExtendedTrustManagerVerifyCallback.verify(ReferenceCountedOpenSslClientContext.java:255)
        at io.netty.handler.ssl.ReferenceCountedOpenSslContext$AbstractCertificateVerifier.verify(ReferenceCountedOpenSslContext.java:701)
        at io.netty.internal.tcnative.SSL.readFromSSL(Native Method)
        at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.readPlaintextData(ReferenceCountedOpenSslEngine.java:594)
        at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.unwrap(ReferenceCountedOpenSslEngine.java:1179)
        at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.unwrap(ReferenceCountedOpenSslEngine.java:1296)
        at io.netty.handler.ssl.ReferenceCountedOpenSslEngine.unwrap(ReferenceCountedOpenSslEngine.java:1339)
        at io.netty.handler.ssl.SslHandler$SslEngineType$1.unwrap(SslHandler.java:206)
        at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1372)
        at io.netty.handler.ssl.SslHandler.decodeNonJdkCompatible(SslHandler.java:1279)
        at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1316)
        at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:501)
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:440)
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
        at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792)
        at io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe$1.run(AbstractEpollChannel.java:387)
        at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
        at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
        at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:384)
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
        at java.base/java.lang.Thread.run(Thread.java:834)

[violetagg]

@estigma88 Please report this in a separate issue as it is not the same problem

[estigma88]

Created #1148