dubbo thrift use problem

[qbmiller]

we use dubbo thrift ;
after running sometime, will come out some problem .
client show timemout error . but provider show 1ms cost; Then we will restart client to solve the problem

we use dubbo 2.6.3

blow is the wrong log

2020-12-31 at 20:00:00.049 [http-nio-8484-exec-6] [WARN ] [com.alibaba.dubbo.rpc.cluster.support.FailoverClusterInvoker::doInvoke:80] - [TRACE: 4d731572f2c7064c] [SPAN: f179bca5642fe2c9] [DUBBO] Although retry the method getUserID in the service com.xxxx.thrift.passport.Passport$Iface was successful by the provider 172.17.0.154:30770, but there have been failed providers [172.17.0.152:30770] (1/3) from the registry 172.17.0.137:2181 on the consumer 172.17.0.154 using the dubbo version 2.6.3.1-SNAPSHOT. Last error is: Waiting server-side response timeout by scan timer. start time: 2020-12-31 19:59:59.507, end time: 2020-12-31 20:00:00.010, client elapsed: 0 ms, server elapsed: 502 ms, timeout: 500 ms, request: Request [id=254956, version=2.0.2, twoway=true, event=false, broken=false, data=RpcInvocation [methodName=getUserID, parameterTypes=[class com.xxxx.thrift.base.Request, class java.lang.String], arguments=[Request(log_id:4d731572f2c7064c-06cb57f46830dfb9-1-f179bca5642fe2c9, ext_json:{}), 639fee6450644054a3b5670b3a827e35], attachments={path=com.xxxx.thrift.passport.Passport$Iface, class.name.generator=dubbo, interface=com.xxxx.thrift.passport.Passport$Iface, timeout=500}]], channel: /172.17.0.154:49806 -> /172.17.0.152:30770, dubbo version: 2.6.3.1-SNAPSHOT, current host: 172.17.0.154 com.alibaba.dubbo.rpc.RpcException: Waiting server-side response timeout by scan timer. start time: 2020-12-31 19:59:59.507, end time: 2020-12-31 20:00:00.010, client elapsed: 0 ms, server elapsed: 502 ms, timeout: 500 ms, request: Request [id=254956, version=2.0.2, twoway=true, event=false, broken=false, data=RpcInvocation [methodName=getUserID, parameterTypes=[class com.xxxx.thrift.base.Request, class java.lang.String], arguments=[Request(log_id:4d731572f2c7064c-06cb57f46830dfb9-1-f179bca5642fe2c9, ext_json:{}), 639fee6450644054a3b5670b3a827e35], attachments={path=com.xxxx.thrift.passport.Passport$Iface, class.name.generator=dubbo, interface=com.xxxx.thrift.passport.Passport$Iface, timeout=500}]], channel: /172.17.0.154:49806 -> /172.17.0.152:30770
at com.alibaba.dubbo.rpc.protocol.thrift.ThriftInvoker.doInvoke(ThriftInvoker.java:90) ~[dubbo-2.6.3.1-SNAPSHOT.jar!/:2.6.3.1-SNAPSHOT]
at com.alibaba.dubbo.rpc.protocol.AbstractInvoker.invoke(AbstractInvoker.java:161) ~[dubbo-2.6.3.1-SNAPSHOT.jar!/:2.6.3.1-SNAPSHOT]
at com.xxxx.axt.filter.DubboBaseFilterClient.invoke(DubboBaseFilterClient.java:51) ~[pico-spring1-starter-web-0.0.4-SNAPSHOT.jar!/:0.0.4-SNAPSHOT]
at com.alibaba.dubbo.rpc.protocol.ProtocolFilterWrapper$1.invoke(ProtocolFilterWrapper.java:72) ~[dubbo-2.6.3.1-SNAPSHOT.jar!/:2.6.3.1-SNAPSHOT]
at com.alibaba.dubbo.monitor.support.MonitorFilter.invoke(MonitorFilter.java:75) ~[dubbo-2.6.3.1-SNAPSHOT.jar!/:2.6.3.1-SNAPSHOT]
at com.alibaba.dubbo.rpc.protocol.ProtocolFilterWrapper$1.invoke(ProtocolFilterWrapper.java:72) ~[dubbo-2.6.3.1-SNAPSHOT.jar!/:2.6.3.1-SNAPSHOT]
at com.alibaba.dubbo.rpc.protocol.dubbo.filter.FutureFilter.invoke(FutureFilter.java:54) ~[dubbo-2.6.3.1-SNAPSHOT.jar!/:2.6.3.1-SNAPSHOT]
at com.alibaba.dubbo.rpc.protocol.ProtocolFilterWrapper$1.invoke(ProtocolFilterWrapper.java:72) ~[dubbo-2.6.3.1-SNAPSHOT.jar!/:2.6.3.1-SNAPSHOT]
at com.alibaba.dubbo.rpc.filter.ConsumerContextFilter.invoke(ConsumerContextFilter.java:49) ~[dubbo-2.6.3.1-SNAPSHOT.jar!/:2.6.3.1-SNAPSHOT]
at com.alibaba.dubbo.rpc.protocol.ProtocolFilterWrapper$1.invoke(ProtocolFilterWrapper.java:72) ~[dubbo-2.6.3.1-SNAPSHOT.jar!/:2.6.3.1-SNAPSHOT]
at com.alibaba.dubbo.rpc.listener.ListenerInvokerWrapper.invoke(ListenerInvokerWrapper.java:77) ~[dubbo-2.6.3.1-SNAPSHOT.jar!/:2.6.3.1-SNAPSHOT]
at com.alibaba.dubbo.rpc.protocol.InvokerWrapper.invoke(InvokerWrapper.java:56) ~[dubbo-2.6.3.1-SNAPSHOT.jar!/:2.6.3.1-SNAPSHOT]
at com.alibaba.dubbo.rpc.cluster.support.FailoverClusterInvoker.doInvoke(FailoverClusterInvoker.java:78) [dubbo-2.6.3.1-SNAPSHOT.jar!/:2.6.3.1-SNAPSHOT]
at com.alibaba.dubbo.rpc.cluster.support.AbstractClusterInvoker.invoke(AbstractClusterInvoker.java:243) [dubbo-2.6.3.1-SNAPSHOT.jar!/:2.6.3.1-SNAPSHOT]
at com.alibaba.dubbo.rpc.cluster.support.wrapper.MockClusterInvoker.invoke(MockClusterInvoker.java:75) [dubbo-2.6.3.1-SNAPSHOT.jar!/:2.6.3.1-SNAPSHOT]
at com.alibaba.dubbo.rpc.cluster.support.AvailableCluster$1.doInvoke(AvailableCluster.java:45) [dubbo-2.6.3.1-SNAPSHOT.jar!/:2.6.3.1-SNAPSHOT]
at com.alibaba.dubbo.rpc.cluster.support.AbstractClusterInvoker.invoke(AbstractClusterInvoker.java:243) [dubbo-2.6.3.1-SNAPSHOT.jar!/:2.6.3.1-SNAPSHOT]
at com.alibaba.dubbo.rpc.cluster.support.wrapper.MockClusterInvoker.invoke(MockClusterInvoker.java:75) [dubbo-2.6.3.1-SNAPSHOT.jar!/:2.6.3.1-SNAPSHOT]
at com.alibaba.dubbo.rpc.proxy.InvokerInvocationHandler.invoke(InvokerInvocationHandler.java:52) [dubbo-2.6.3.1-SNAPSHOT.jar!/:2.6.3.1-SNAPSHOT]
at com.alibaba.dubbo.common.bytecode.proxy3.getUserID(proxy3.java) [dubbo-2.6.3.1-SNAPSHOT.jar!/:2.6.3.1-SNAPSHOT]
at com.xxxx.axt.filter.BaUtil.initCurrentLoginUser(BaUtil.java:83) [pico-spring1-starter-web-0.0.4-SNAPSHOT.jar!/:0.0.4-SNAPSHOT]
at com.xxxx.axt.filter.BaTraceLogFilter.doFilterInternal(BaTraceLogFilter.java:82) [pico-spring1-starter-web-0.0.4-SNAPSHOT.jar!/:0.0.4-SNAPSHOT]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.25.RELEASE.jar!/:4.3.25.RELEASE]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.43.jar!/:8.5.43]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.43.jar!/:8.5.43]
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:197) [spring-web-4.3.25.RELEASE.jar!/:4.3.25.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.25.RELEASE.jar!/:4.3.25.RELEASE]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.43.jar!/:8.5.43]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.43.jar!/:8.5.43]
at org.springframework.boot.actuate.autoconfigure.MetricsFilter.doFilterInternal(MetricsFilter.java:100) [spring-boot-actuator-1.5.22.RELEASE.jar!/:1.5.22.RELEASE]
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-4.3.25.RELEASE.jar!/:4.3.25.RELEASE]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.43.jar!/:8.5.43]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.43.jar!/:8.5.43]
at brave.servlet.TracingFilter.doFilter(TracingFilter.java:87) [brave-instrumentation-servlet-5.12.7.jar!/:?]
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.43.jar!/:8.5.43]
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.43.jar!/:8.5.43]
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199) [tomcat-embed-core-8.5.43.jar!/:8.5.43]
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) [tomcat-embed-core-8.5.43.jar!/:8.5.43]
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:493) [tomcat-embed-core-8.5.43.jar!/:8.5.43]
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:137) [tomcat-embed-core-8.5.43.jar!/:8.5.43]
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81) [tomcat-embed-core-8.5.43.jar!/:8.5.43]
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) [tomcat-embed-core-8.5.43.jar!/:8.5.43]
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343) [tomcat-embed-core-8.5.43.jar!/:8.5.43]
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:798) [tomcat-embed-core-8.5.43.jar!/:8.5.43]
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) [tomcat-embed-core-8.5.43.jar!/:8.5.43]
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:808) [tomcat-embed-core-8.5.43.jar!/:8.5.43]
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1498) [tomcat-embed-core-8.5.43.jar!/:8.5.43]
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-embed-core-8.5.43.jar!/:8.5.43]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_20]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_20]
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-8.5.43.jar!/:8.5.43]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_20]
Caused by: com.alibaba.dubbo.remoting.TimeoutException: Waiting server-side response timeout by scan timer. start time: 2020-12-31 19:59:59.507, end time: 2020-12-31 20:00:00.010, client elapsed: 0 ms, server elapsed: 502 ms, timeout: 500 ms, request: Request [id=254956, version=2.0.2, twoway=true, event=false, broken=false, data=RpcInvocation [methodName=getUserID, parameterTypes=[class com.xxxx.thrift.base.Request, class java.lang.String], arguments=[Request(log_id:4d731572f2c7064c-06cb57f46830dfb9-1-f179bca5642fe2c9, ext_json:{}), 639fee6450644054a3b5670b3a827e35], attachments={path=com.xxxx.thrift.passport.Passport$Iface, class.name.generator=dubbo, interface=com.xxxx.thrift.passport.Passport$Iface, timeout=500}]], channel: /172.17.0.154:49806 -> /172.17.0.152:30770
at com.alibaba.dubbo.remoting.exchange.support.DefaultFuture.returnFromResponse(DefaultFuture.java:220) ~[dubbo-2.6.3.1-SNAPSHOT.jar!/:2.6.3.1-SNAPSHOT]
at com.alibaba.dubbo.remoting.exchange.support.DefaultFuture.get(DefaultFuture.java:139) ~[dubbo-2.6.3.1-SNAPSHOT.jar!/:2.6.3.1-SNAPSHOT]
at com.alibaba.dubbo.remoting.exchange.support.DefaultFuture.get(DefaultFuture.java:112) ~[dubbo-2.6.3.1-SNAPSHOT.jar!/:2.6.3.1-SNAPSHOT]
at com.alibaba.dubbo.rpc.protocol.thrift.ThriftInvoker.doInvoke(ThriftInvoker.java:87) ~[dubbo-2.6.3.1-SNAPSHOT.jar!/:2.6.3.1-SNAPSHOT]
... 51 more

[dkisser]

just warn ,it doesn't matter.provider (172.17.0.152:30770 ) response timeout, but another provider(172.17.0.154:30770) response successfully.if you want know the reason of the timeout ,you should watch the log of the provder(172.17.0.152:30770) .

[CrazyHZM]

The above answer should solve your problem, temporarily close the issue, if there is still a problem, you can reopen it