-
Notifications
You must be signed in to change notification settings - Fork 3.9k
Open
Description
What version of gRPC-Java are you using?
gRPC-Java 1.75
What is your environment?
Linux arm64,JDK 17.0.15
What did you expect to see?
The blocking stub should return once the server sends a grpc-status: 0 completion, or throw a timeout exception if the deadline expires — it should never hang indefinitely.
What did you see instead?
When using a blocking stub with a timeout interceptor, the client occasionally hangs forever in:
io.grpc.stub.ClientCalls$ThreadlessExecutor.waitAndDrain(ClientCalls.java:817)
even though the server has already completed the RPC successfully (grpc-status: 0).
The thread remains parked indefinitely, and the application never proceeds.
This issue seems to occur randomly (low frequency)
Steps to reproduce the bug
###nterceptor
public class TimeoutInterceptor implements ClientInterceptor {
private final long timeout;
/**
* @param timeout ms
*/
public TimeoutInterceptor(long timeout) {
this.timeout = timeout;
}
@Override
public <ReqT, RespT> ClientCall<ReqT, RespT> interceptCall(
MethodDescriptor<ReqT, RespT> method,
CallOptions callOptions,
Channel next) {
return next.newCall(method,
callOptions.withDeadlineAfter(timeout, TimeUnit.MILLISECONDS));
}
}stack
"Test worker" #1 prio=5 os_prio=0 cpu=68854.65ms elapsed=51528.48s tid=0x0000ffffb40311a0 nid=0x321d1c waiting on condition [0x0000ffffb99fb000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@17.0.15/Native Method)
- parking to wait for <0x00000000d69d5a18> (a io.grpc.stub.ClientCalls$ThreadlessExecutor)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:211)
at io.grpc.stub.ClientCalls$ThreadlessExecutor.waitAndDrain(ClientCalls.java:817)
at io.grpc.stub.ClientCalls.blockingUnaryCall(ClientCalls.java:166)
at org.tron.api.DatabaseGrpc$DatabaseBlockingStub.getBlockByNum(DatabaseGrpc.java:343)
at org.tron.common.client.DatabaseGrpcClient.getBlock(DatabaseGrpcClient.java:44)
...
grpc log
12:45:36.755 DEBUG [i.g.n.NettyClientHandler] [id: 0xe05d631e, L:/127.0.0.1:48716 - R:/127.0.0.1:57932] OUTBOUND HEADERS: streamId=5 headers=GrpcHttp2OutboundHeaders[:authority: 127.0.0.1:57932, :path: /protocol.Database/GetBlockByNum, :method: POST, :scheme: http, content-type: application/grpc, te: trailers, user-agent: grpc-java-netty/1.75.0, grpc-accept-encoding: gzip] padding=0 endStream=false
12:45:36.756 DEBUG [i.g.n.NettyClientHandler] [id: 0xe05d631e, L:/127.0.0.1:48716 - R:/127.0.0.1:57932] OUTBOUND DATA: streamId=5 padding=0 endStream=true length=5 bytes=0000000000
12:45:36.756 DEBUG [i.g.n.NettyServerHandler] [id: 0xa59d39e1, L:/127.0.0.1:57932 - R:/127.0.0.1:48716] INBOUND HEADERS: streamId=5 headers=GrpcHttp2RequestHeaders[:path: /protocol.Database/GetBlockByNum, :authority: 127.0.0.1:57932, :method: POST, :scheme: http, te: trailers, content-type: application/grpc, user-agent: grpc-java-netty/1.75.0, grpc-accept-encoding: gzip] padding=0 endStream=false
12:45:36.757 DEBUG [i.g.n.NettyServerHandler] [id: 0xa59d39e1, L:/127.0.0.1:57932 - R:/127.0.0.1:48716] INBOUND DATA: streamId=5 padding=0 endStream=true length=5 bytes=0000000000
12:45:36.758 DEBUG [i.g.n.NettyServerHandler] [id: 0xa59d39e1, L:/127.0.0.1:57932 - R:/127.0.0.1:48716] OUTBOUND HEADERS: streamId=5 headers=GrpcHttp2OutboundHeaders[:status: 200, content-type: application/grpc, grpc-encoding: identity, grpc-accept-encoding: gzip] padding=0 endStream=false
12:45:36.759 DEBUG [i.g.n.NettyServerHandler] [id: 0xa59d39e1, L:/127.0.0.1:57932 - R:/127.0.0.1:48716] OUTBOUND DATA: streamId=5 padding=0 endStream=false length=663 bytes=00000002920a720a705a6e0801126a0a2d747970652e676f6f676c65617069732e636f6d2f70726f746f636f6c2e5472616e73666572436f6e74726163741239...
12:45:36.759 DEBUG [i.g.n.NettyServerHandler] [id: 0xa59d39e1, L:/127.0.0.1:57932 - R:/127.0.0.1:48716] OUTBOUND HEADERS: streamId=5 headers=GrpcHttp2OutboundHeaders[grpc-status: 0] padding=0 endStream=true
12:45:36.760 DEBUG [i.g.n.NettyClientHandler] [id: 0xe05d631e, L:/127.0.0.1:48716 - R:/127.0.0.1:57932] INBOUND HEADERS: streamId=5 headers=GrpcHttp2ResponseHeaders[:status: 200, content-type: application/grpc, grpc-encoding: identity, grpc-accept-encoding: gzip] padding=0 endStream=false
12:45:36.760 DEBUG [i.g.n.NettyClientHandler] [id: 0xe05d631e, L:/127.0.0.1:48716 - R:/127.0.0.1:57932] INBOUND DATA: streamId=5 padding=0 endStream=false length=663 bytes=00000002920a720a705a6e0801126a0a2d747970652e676f6f676c65617069732e636f6d2f70726f746f636f6c2e5472616e73666572436f6e74726163741239...
12:45:36.761 DEBUG [i.g.n.NettyClientHandler] [id: 0xe05d631e, L:/127.0.0.1:48716 - R:/127.0.0.1:57932] INBOUND HEADERS: streamId=5 headers=GrpcHttp2ResponseHeaders[grpc-status: 0] padding=0 endStream=true
12:46:36.790 DEBUG [i.g.n.NettyServerHandler] [id: 0xa59d39e1, L:/127.0.0.1:57932 - R:/127.0.0.1:48716] OUTBOUND GO_AWAY: lastStreamId=2147483647 errorCode=0 length=8 bytes=6d61785f69646c65
12:46:36.791 DEBUG [i.g.n.NettyServerHandler] [id: 0xa59d39e1, L:/127.0.0.1:57932 - R:/127.0.0.1:48716] OUTBOUND PING: ack=false bytes=40715087873
12:46:36.791 DEBUG [i.g.n.NettyClientHandler] [id: 0xe05d631e, L:/127.0.0.1:48716 - R:/127.0.0.1:57932] INBOUND GO_AWAY: lastStreamId=2147483647 errorCode=0 length=8 bytes=6d61785f69646c65
12:46:36.795 DEBUG [i.g.n.NettyClientHandler] [id: 0xe05d631e, L:/127.0.0.1:48716 - R:/127.0.0.1:57932] INBOUND PING: ack=false bytes=40715087873
12:46:36.795 DEBUG [i.g.n.NettyClientHandler] [id: 0xe05d631e, L:/127.0.0.1:48716 - R:/127.0.0.1:57932] OUTBOUND PING: ack=true bytes=40715087873
12:46:36.795 DEBUG [i.g.n.NettyServerHandler] [id: 0xa59d39e1, L:/127.0.0.1:57932 - R:/127.0.0.1:48716] INBOUND PING: ack=true bytes=40715087873
12:46:36.795 DEBUG [i.g.n.NettyServerHandler] [id: 0xa59d39e1, L:/127.0.0.1:57932 - R:/127.0.0.1:48716] OUTBOUND GO_AWAY: lastStreamId=5 errorCode=0 length=8 bytes=6d61785f69646c65
12:46:36.796 DEBUG [i.g.n.NettyClientHandler] [id: 0xe05d631e, L:/127.0.0.1:48716 - R:/127.0.0.1:57932] INBOUND GO_AWAY: lastStreamId=5 errorCode=0 length=8 bytes=6d61785f69646c65
Metadata
Metadata
Assignees
Labels
No labels