Connection reset by peer; nested exception is io.netty.channel.unix.Errors$NativeIoException: readAddress(..)

[zxltmjzyt]

2024-03-25 10:14:45.508 DEBUG 1 --- [or-http-epoll-2] r.n.http.server.HttpServerOperations     : [1befee4a, L:/172.20.83.253:8080 - R:/172.20.245.192:57904] New http connection, requesting read
2024-03-25 10:14:45.508 DEBUG 1 --- [or-http-epoll-3] r.n.http.server.HttpServerOperations     : [e0a90d1a, L:/172.20.83.253:8080 - R:/172.20.245.192:57906] New http connection, requesting read
2024-03-25 10:14:45.508 DEBUG 1 --- [or-http-epoll-2] reactor.netty.transport.TransportConfig  : [1befee4a, L:/172.20.83.253:8080 - R:/172.20.245.192:57904] Initialized pipeline DefaultChannelPipeline{(reactor.left.httpCodec = io.netty.handler.codec.http.HttpServerCodec), (reactor.left.httpTrafficHandler = reactor.netty.http.server.HttpTrafficHandler), (reactor.right.reactiveBridge = reactor.netty.channel.ChannelOperationsHandler)}
2024-03-25 10:14:45.508 DEBUG 1 --- [or-http-epoll-3] reactor.netty.transport.TransportConfig  : [e0a90d1a, L:/172.20.83.253:8080 - R:/172.20.245.192:57906] Initialized pipeline DefaultChannelPipeline{(reactor.left.httpCodec = io.netty.handler.codec.http.HttpServerCodec), (reactor.left.httpTrafficHandler = reactor.netty.http.server.HttpTrafficHandler), (reactor.right.reactiveBridge = reactor.netty.channel.ChannelOperationsHandler)}
2024-03-25 10:14:45.509 DEBUG 1 --- [or-http-epoll-3] r.n.http.server.HttpServerOperations     : [e0a90d1a, L:/172.20.83.253:8080 - R:/172.20.245.192:57906] Increasing pending responses, now 1
2024-03-25 10:14:45.509 DEBUG 1 --- [or-http-epoll-2] r.n.http.server.HttpServerOperations     : [1befee4a, L:/172.20.83.253:8080 - R:/172.20.245.192:57904] Increasing pending responses, now 1
2024-03-25 10:14:45.509 DEBUG 1 --- [or-http-epoll-3] reactor.netty.http.server.HttpServer     : [e0a90d1a-1, L:/172.20.83.253:8080 - R:/172.20.245.192:57906] Handler is being applied: org.springframework.http.server.reactive.ReactorHttpHandlerAdapter@3c20eb9e
2024-03-25 10:14:45.509 DEBUG 1 --- [or-http-epoll-2] reactor.netty.http.server.HttpServer     : [1befee4a-1, L:/172.20.83.253:8080 - R:/172.20.245.192:57904] Handler is being applied: org.springframework.http.server.reactive.ReactorHttpHandlerAdapter@3c20eb9e
2024-03-25 10:14:45.510 DEBUG 1 --- [dedElastic-3387] c.s.gateway.filter.AuthenticateFilter    : 请求进入网关，path：/gateway/hyjg-subplatform-service/webApi/getPermissionMenu
2024-03-25 10:14:45.510 DEBUG 1 --- [dedElastic-3401] c.s.gateway.filter.AuthenticateFilter    : 请求进入网关，path：/gateway/hyjg-portal-service/summarytasks/getMessageList
2024-03-25 10:14:45.511 DEBUG 1 --- [or-http-epoll-1] r.n.resources.PooledConnectionProvider   : [5516f841, L:/172.20.83.253:57848 - R:gateway-midplat-tech-service.ddplatform.svc.cluster.local/10.68.105.59:8080] Channel acquired, now: 1 active connections, 3 inactive connections and 0 pending acquire requests.
2024-03-25 10:14:45.511 DEBUG 1 --- [or-http-epoll-3] r.n.resources.PooledConnectionProvider   : [6d5048d9, L:/172.20.83.253:49938 - R:gateway-midplat-tech-service.ddplatform.svc.cluster.local/10.68.105.59:8080] Channel acquired, now: 2 active connections, 2 inactive connections and 0 pending acquire requests.
2024-03-25 10:14:45.511 DEBUG 1 --- [or-http-epoll-1] r.netty.http.client.HttpClientConnect    : [5516f841-13, L:/172.20.83.253:57848 - R:gateway-midplat-tech-service.ddplatform.svc.cluster.local/10.68.105.59:8080] Handler is being applied: {uri=http://gateway-midplat-tech-service.ddplatform.svc.cluster.local:8080/platform/gateway/account/accountRestService/getToken, method=POST}
2024-03-25 10:14:45.511 DEBUG 1 --- [or-http-epoll-3] r.netty.http.client.HttpClientConnect    : [6d5048d9-14, L:/172.20.83.253:49938 - R:gateway-midplat-tech-service.ddplatform.svc.cluster.local/10.68.105.59:8080] Handler is being applied: {uri=http://gateway-midplat-tech-service.ddplatform.svc.cluster.local:8080/platform/gateway/account/accountRestService/getToken, method=POST}
2024-03-25 10:14:45.511 DEBUG 1 --- [or-http-epoll-1] r.n.r.DefaultPooledConnectionProvider    : [5516f841-13, L:/172.20.83.253:57848 - R:gateway-midplat-tech-service.ddplatform.svc.cluster.local/10.68.105.59:8080] onStateChange(POST{uri=/platform/gateway/account/accountRestService/getToken, connection=PooledConnection{channel=[id: 0x5516f841, L:/172.20.83.253:57848 - R:gateway-midplat-tech-service.ddplatform.svc.cluster.local/10.68.105.59:8080]}}, [request_prepared])
2024-03-25 10:14:45.511 DEBUG 1 --- [or-http-epoll-3] r.n.r.DefaultPooledConnectionProvider    : [6d5048d9-14, L:/172.20.83.253:49938 - R:gateway-midplat-tech-service.ddplatform.svc.cluster.local/10.68.105.59:8080] onStateChange(POST{uri=/platform/gateway/account/accountRestService/getToken, connection=PooledConnection{channel=[id: 0x6d5048d9, L:/172.20.83.253:49938 - R:gateway-midplat-tech-service.ddplatform.svc.cluster.local/10.68.105.59:8080]}}, [request_prepared])
2024-03-25 10:14:45.512 DEBUG 1 --- [or-http-epoll-1] r.n.r.DefaultPooledConnectionProvider    : [5516f841-13, L:/172.20.83.253:57848 - R:gateway-midplat-tech-service.ddplatform.svc.cluster.local/10.68.105.59:8080] onStateChange(POST{uri=/platform/gateway/account/accountRestService/getToken, connection=PooledConnection{channel=[id: 0x5516f841, L:/172.20.83.253:57848 - R:gateway-midplat-tech-service.ddplatform.svc.cluster.local/10.68.105.59:8080]}}, [request_sent])
2024-03-25 10:14:45.512 DEBUG 1 --- [or-http-epoll-3] r.n.r.DefaultPooledConnectionProvider    : [6d5048d9-14, L:/172.20.83.253:49938 - R:gateway-midplat-tech-service.ddplatform.svc.cluster.local/10.68.105.59:8080] onStateChange(POST{uri=/platform/gateway/account/accountRestService/getToken, connection=PooledConnection{channel=[id: 0x6d5048d9, L:/172.20.83.253:49938 - R:gateway-midplat-tech-service.ddplatform.svc.cluster.local/10.68.105.59:8080]}}, [request_sent])
2024-03-25 10:14:45.512  WARN 1 --- [or-http-epoll-1] r.netty.http.client.HttpClientConnect    : [5516f841-13, L:/172.20.83.253:57848 - R:gateway-midplat-tech-service.ddplatform.svc.cluster.local/10.68.105.59:8080] The connection observed an error, the request cannot be retried as the headers/body were sent

io.netty.channel.unix.Errors$NativeIoException: readAddress(..) failed: Connection reset by peer

2024-03-25 10:14:45.512  WARN 1 --- [or-http-epoll-3] r.netty.http.client.HttpClientConnect    : [6d5048d9-14, L:/172.20.83.253:49938 - R:gateway-midplat-tech-service.ddplatform.svc.cluster.local/10.68.105.59:8080] The connection observed an error, the request cannot be retried as the headers/body were sent

io.netty.channel.unix.Errors$NativeIoException: readAddress(..) failed: Connection reset by peer

2024-03-25 10:14:45.513 ERROR 1 --- [or-http-epoll-1] c.s.g.handler.GlobalExceptionHandler     : 网关错误

org.springframework.web.reactive.function.client.WebClientRequestException: readAddress(..) failed: Connection reset by peer; nested exception is io.netty.channel.unix.Errors$NativeIoException: readAddress(..) failed: Connection reset by peer
	at org.springframework.web.reactive.function.client.ExchangeFunctions$DefaultExchangeFunction.lambda$wrapException$9(ExchangeFunctions.java:141)
	Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: 
Error has been observed at the following site(s):
	*__checkpoint ⇢ Request to POST http://gateway-midplat-tech-service.ddplatform.svc.cluster.local:8080/platform/gateway/account/accountRestService/getToken [DefaultWebClient]
	*__checkpoint ⇢ org.springframework.cloud.gateway.filter.WeightCalculatorWebFilter [DefaultWebFilterChain]
	*__checkpoint ⇢ org.springframework.boot.actuate.metrics.web.reactive.server.MetricsWebFilter [DefaultWebFilterChain]
	*__checkpoint ⇢ HTTP GET "/gateway/hyjg-portal-service/summarytasks/getMessageList?currentPage=1&pageSize=3&readStatus=UN_READ&msgType=TEXT" [ExceptionHandlingWebHandler]
Original Stack Trace:
		at org.springframework.web.reactive.function.client.ExchangeFunctions$DefaultExchangeFunction.lambda$wrapException$9(ExchangeFunctions.java:141)
		at reactor.core.publisher.MonoErrorSupplied.subscribe(MonoErrorSupplied.java:55)
		at reactor.core.publisher.Mono.subscribe(Mono.java:4400)
		at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:103)
		at reactor.core.publisher.FluxPeek$PeekSubscriber.onError(FluxPeek.java:222)
		at reactor.core.publisher.FluxPeek$PeekSubscriber.onError(FluxPeek.java:222)
		at reactor.core.publisher.FluxPeek$PeekSubscriber.onError(FluxPeek.java:222)
		at reactor.core.publisher.MonoNext$NextSubscriber.onError(MonoNext.java:93)
		at reactor.core.publisher.MonoFlatMapMany$FlatMapManyMain.onError(MonoFlatMapMany.java:204)
		at reactor.core.publisher.SerializedSubscriber.onError(SerializedSubscriber.java:124)
		at reactor.core.publisher.FluxRetryWhen$RetryWhenMainSubscriber.whenError(FluxRetryWhen.java:225)
		at reactor.core.publisher.FluxRetryWhen$RetryWhenOtherSubscriber.onError(FluxRetryWhen.java:274)
		at reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.drain(FluxConcatMap.java:414)
		at reactor.core.publisher.FluxConcatMap$ConcatMapImmediate.onNext(FluxConcatMap.java:251)
		at reactor.core.publisher.EmitterProcessor.drain(EmitterProcessor.java:491)
		at reactor.core.publisher.EmitterProcessor.tryEmitNext(EmitterProcessor.java:299)
		at reactor.core.publisher.SinkManySerialized.tryEmitNext(SinkManySerialized.java:100)
		at reactor.core.publisher.InternalManySink.emitNext(InternalManySink.java:27)
		at reactor.core.publisher.FluxRetryWhen$RetryWhenMainSubscriber.onError(FluxRetryWhen.java:190)
		at reactor.core.publisher.MonoCreate$DefaultMonoSink.error(MonoCreate.java:194)
		at reactor.netty.http.client.HttpClientConnect$HttpObserver.onUncaughtException(HttpClientConnect.java:384)
		at reactor.netty.ReactorNetty$CompositeConnectionObserver.onUncaughtException(ReactorNetty.java:664)
		at reactor.netty.resources.DefaultPooledConnectionProvider$DisposableAcquire.onUncaughtException(DefaultPooledConnectionProvider.java:219)
		at reactor.netty.resources.DefaultPooledConnectionProvider$PooledConnection.onUncaughtException(DefaultPooledConnectionProvider.java:467)
		at reactor.netty.channel.FluxReceive.drainReceiver(FluxReceive.java:232)
		at reactor.netty.channel.FluxReceive.onInboundError(FluxReceive.java:453)
		at reactor.netty.channel.ChannelOperations.onInboundError(ChannelOperations.java:488)
		at reactor.netty.channel.ChannelOperationsHandler.exceptionCaught(ChannelOperationsHandler.java:126)
		at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302)
		at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281)
		at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:273)
		at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireExceptionCaught(CombinedChannelDuplexHandler.java:424)
		at io.netty.channel.ChannelHandlerAdapter.exceptionCaught(ChannelHandlerAdapter.java:92)
		at io.netty.channel.CombinedChannelDuplexHandler$1.fireExceptionCaught(CombinedChannelDuplexHandler.java:145)
		at io.netty.channel.ChannelInboundHandlerAdapter.exceptionCaught(ChannelInboundHandlerAdapter.java:143)
		at io.netty.channel.CombinedChannelDuplexHandler.exceptionCaught(CombinedChannelDuplexHandler.java:231)
		at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302)
		at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281)
		at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:273)
		at io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1377)
		at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302)
		at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281)
		at io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:907)
		at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.handleReadException(AbstractEpollStreamChannel.java:728)
		at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:821)
		at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:480)
		at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378)
		at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986)
		at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
		at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
		at java.lang.Thread.run(Thread.java:748)
Caused by: io.netty.channel.unix.Errors$NativeIoException: readAddress(..) failed: Connection reset by peer

2024-03-25 10:14:45.514 DEBUG 1 --- [or-http-epoll-1] reactor.netty.ReactorNetty               : [e0a90d1a-1, L:/172.20.83.253:8080 - R:/172.20.245.192:57906] Added encoder [reactor.left.compressionHandler] at the beginning of the user pipeline, full pipeline: [reactor.left.httpCodec, reactor.left.httpTrafficHandler, reactor.left.compressionHandler, reactor.right.reactiveBridge, DefaultChannelPipeline$TailContext#0]
2024-03-25 10:14:45.515 ERROR 1 --- [or-http-epoll-3] c.s.g.handler.GlobalExceptionHandler     : 网关错误

This issue occurs when the page is just loaded, and if you refresh the page, there is no problem.

please help me~

[violetagg]

@zxltmjzyt Please provide tcpdump, also check this FAQ page https://projectreactor.io/docs/netty/release/reference/index.html#faq.connection-closed

[github-actions]

If you would like us to look at this issue, please provide the requested information. If the information is not provided within the next 7 days this issue will be closed.