Intermittent `MongoSocketWriteException` using the reactive driver

[ashishkujoy]

Hi team, we are facing intermittent MongoSocketWriteException

Click to see error stacktrace

at reactor.core.publisher.FluxDoOnEach$DoOnEachSubscriber.onError(FluxDoOnEach.java:186)
   at reactor.core.publisher.FluxDoOnEach$DoOnEachSubscriber.onError(FluxDoOnEach.java:195)
   at reactor.core.publisher.MonoNext$NextSubscriber.onError(MonoNext.java:93)
   at reactor.core.publisher.FluxUsingWhen$UsingWhenSubscriber.deferredError(FluxUsingWhen.java:411)
   at reactor.core.publisher.FluxUsingWhen$RollbackInner.onComplete(FluxUsingWhen.java:488)
   at reactor.core.publisher.Operators.complete(Operators.java:136)
   at reactor.core.publisher.MonoEmpty.subscribe(MonoEmpty.java:45)
   at reactor.core.publisher.Mono.subscribe(Mono.java:4150)
   at reactor.core.publisher.FluxUsingWhen$UsingWhenSubscriber.onError(FluxUsingWhen.java:377)
   at reactor.core.publisher.MonoFlatMap$FlatMapMain.onError(MonoFlatMap.java:172)
   at reactor.core.publisher.MonoCollectList$MonoCollectListSubscriber.onError(MonoCollectList.java:113)
   at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:106)
   at reactor.core.publisher.Operators.error(Operators.java:197)
   at reactor.core.publisher.MonoError.subscribe(MonoError.java:52)
   at reactor.core.publisher.Mono.subscribe(Mono.java:4150)
   at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:103)
   at reactor.core.publisher.MonoFlatMapMany$FlatMapManyInner.onError(MonoFlatMapMany.java:255)
   at reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.onError(FluxConcatMap.java:262)
   at com.mongodb.reactivestreams.client.internal.AbstractSubscription.onError(AbstractSubscription.java:142)
   at com.mongodb.reactivestreams.client.internal.MongoIterableSubscription.lambda$requestInitialData$0(MongoIterableSubscription.java:47)
   at com.mongodb.internal.async.ErrorHandlingResultCallback.onResult(ErrorHandlingResultCallback.java:48)
   at com.mongodb.internal.async.client.OperationExecutorImpl$1$1$1.onResult(OperationExecutorImpl.java:92)
   at com.mongodb.internal.async.ErrorHandlingResultCallback.onResult(ErrorHandlingResultCallback.java:48)
   at com.mongodb.internal.operation.FindOperation$3.onResult(FindOperation.java:750)
   at com.mongodb.internal.operation.OperationHelper$ReferenceCountedReleasingWrappedCallback.onResult(OperationHelper.java:531)
   at com.mongodb.internal.operation.CommandOperationHelper$TransformingReadResultCallback.onResult(CommandOperationHelper.java:543)
   at com.mongodb.internal.async.ErrorHandlingResultCallback.onResult(ErrorHandlingResultCallback.java:48)
   at com.mongodb.internal.connection.DefaultServer$DefaultServerProtocolExecutor$2.onResult(DefaultServer.java:285)
   at com.mongodb.internal.async.ErrorHandlingResultCallback.onResult(ErrorHandlingResultCallback.java:48)
   at com.mongodb.internal.connection.CommandProtocolImpl$1.onResult(CommandProtocolImpl.java:82)
   at com.mongodb.internal.connection.DefaultConnectionPool$PooledConnection$2.onResult(DefaultConnectionPool.java:530)
   at com.mongodb.internal.connection.UsageTrackingInternalConnection$2.onResult(UsageTrackingInternalConnection.java:142)
   at com.mongodb.internal.async.ErrorHandlingResultCallback.onResult(ErrorHandlingResultCallback.java:48)
   at com.mongodb.internal.connection.InternalStreamConnection$2.onResult(InternalStreamConnection.java:435)
   at com.mongodb.internal.connection.InternalStreamConnection$2.onResult(InternalStreamConnection.java:429)
   at com.mongodb.internal.async.ErrorHandlingResultCallback.onResult(ErrorHandlingResultCallback.java:48)
   at com.mongodb.internal.connection.InternalStreamConnection$3.failed(InternalStreamConnection.java:542)
   at com.mongodb.connection.netty.NettyStream$2.operationComplete(NettyStream.java:204)
   at com.mongodb.connection.netty.NettyStream$2.operationComplete(NettyStream.java:200)
   at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:578)
   at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:552)
   at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:491)
   at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:616)
   at io.netty.util.concurrent.DefaultPromise.setFailure0(DefaultPromise.java:609)
   at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:117)
   at io.netty.channel.AbstractChannel$AbstractUnsafe.safeSetFailure(AbstractChannel.java:1021)
   at io.netty.channel.AbstractChannel$AbstractUnsafe.write(AbstractChannel.java:882)
   at io.netty.channel.DefaultChannelPipeline$HeadContext.write(DefaultChannelPipeline.java:1367)
   at io.netty.channel.AbstractChannelHandlerContext.invokeWrite0(AbstractChannelHandlerContext.java:717)
   at io.netty.channel.AbstractChannelHandlerContext.invokeWriteAndFlush(AbstractChannelHandlerContext.java:764)
   at io.netty.channel.AbstractChannelHandlerContext$WriteTask.run(AbstractChannelHandlerContext.java:1071)
   at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)
   at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)
   at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)
   at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
   at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
   at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
   at java.base/java.lang.Thread.run(Thread.java:829)\nCaused by: org.springframework.dao.DataAccessResourceFailureException: Exception sending message; nested exception is com.mongodb.MongoSocketWriteException: Exception sending message
   at org.springframework.data.mongodb.core.MongoExceptionTranslator.translateExceptionIfPossible(MongoExceptionTranslator.java:85)
   at org.springframework.data.mongodb.core.ReactiveMongoTemplate.potentiallyConvertRuntimeException(ReactiveMongoTemplate.java:2827)
   at org.springframework.data.mongodb.core.ReactiveMongoTemplate.lambda$translateException$91(ReactiveMongoTemplate.java:2810)
   at reactor.core.publisher.Flux.lambda$onErrorMap$28(Flux.java:6677)
   at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:94)
   ... 42 common frames omitted\nCaused by: com.mongodb.MongoSocketWriteException: Exception sending message
   at com.mongodb.internal.connection.InternalStreamConnection.translateWriteException(InternalStreamConnection.java:619)
   at com.mongodb.internal.connection.InternalStreamConnection.access$1200(InternalStreamConnection.java:78)
   ... 22 common frames omitted\nCaused by: io.netty.channel.StacklessClosedChannelException: null
   at io.netty.channel.AbstractChannel$AbstractUnsafe.write(Object, ChannelPromise)(Unknown Source)\n"}

Version details:

org.springframework.boot:spring-boot-starter-data-mongodb-reactive -> 2.4.5
org.springframework.data:spring-data-mongodb:3.1.8
org.springframework.boot:spring-boot-starter-actuator -> 2.4.5
org.jetbrains.kotlin:kotlin-stdlib-jdk8:1.4.32

Can you please help us understand what could be the reason for this intermittent failure.

[christophstrobl]

Can you please provide information on the context in which you execute the operation, the op itself, etc.
A complete minimal sample (something that we can unzip or git clone, build, and deploy) that reproduces the problem would be very helpful to spot the cause. //cc @jyemin & @rozza

[ashishkujoy]

Thanks @christophstrobl for quick reply. I will try to provide a sample application.

[rozza]

Could you provide the MongoSocketWriteException details as well? There should be some pertinent information there as to the cause.

[ashishkujoy]

@rozza this is full error stack from the exception.

[ashishkujoy]

Repo to show how we are using the mongo, we may not able to reproduce the issue with this sample code, this only for showing how we are using and versions we are using.
In our use case we are only reading from DB, there is now write from this application, we have a separate application which add data in db.

NOTE: In production we are using AWS provided Document DB.

[mp911de]

This ticket isn't actionable for us as I/O exceptions relate to the driver and the attached reproducer uses default driver settings.