Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Question] After crashing the runtime and restarting it, the grpc client will not be able to reconnect by sending heartbeat packets. #4587

Closed
1 task done
hhuang1231 opened this issue Nov 30, 2023 · 6 comments
Assignees
Labels
question Further information is requested

Comments

@hhuang1231
Copy link
Contributor

hhuang1231 commented Nov 30, 2023

Search before asking

  • I had searched in the issues and found no similar issues.

Question

The connection between the client and the runtime can be maintained through heartbeat detection. For http and tcp, when the runtime crashes and then restarts, the connection can be reacquired through the heartbeat packet, but for grcp, once the runtime crashes, io.grpc.stub.StreamObserver will be marked as completed. At this time, the client will no longer be able to send heartbeat packets, and will not be able to reconnect to the runtime.

Please tell me if this is a bug?


通过心跳检测可以保持client与runtime的连接。对于http和tcp来说,当runtime宕机再重启后,通过心跳包可以重新获取连接,但是对于grcp来说,一旦runtime宕机就会标记io.grpc.stub.StreamObserver为completed。此时client将无法再发送心跳包,进而无法重新连接runtime。

请问这是否是个bug?

image

image
image

@hhuang1231 hhuang1231 added the question Further information is requested label Nov 30, 2023
@hhuang1231
Copy link
Contributor Author

I also want to know why org.apache.eventmesh.client.grpc.consumer.EventMeshGrpcConsumer#resubscribe is limited to using subStreamHandler for sending.
我还想知道org.apache.eventmesh.client.grpc.consumer.EventMeshGrpcConsumer#resubscribe为什么限定使用subStreamHandler进行发送。

image
As far as I know, there are only two ways to create a subStreamHandler:

  1. Call public void subscribe(final List<SubscriptionItem> subscriptionItems) to automatically create
  2. Call the setter
    For the startup method of grpc springboot in examples, subStreamHandler is not created, so NPE will be thrown when resubscribe is called.

我目前所知创建subStreamHandler应该只有两种方式:

  1. 调用public void subscribe(final List<SubscriptionItem> subscriptionItems)自动创建
  2. 调用setter

而对于examples中grpc springboot的启动方式来说,并没有创建subStreamHandler,因此当调用resubscribe时将会抛出NPE

@pandaapo
Copy link
Member

but for grcp, once the runtime crashes, io.grpc.stub.StreamObserver will be marked as completed. At this time, the client will no longer be able to send heartbeat packets, and will not be able to reconnect to the runtime.

About the first of the two questions you asked, calling the #onCompleted() method in StreamObserver#onError() generally does not result in reconnection failure. This logic only indicates that there is a problem with the server causing the stream to be interrupted, and the client directly processes it as a received response. Under normal circumstances, gPRC supports reconnection by default. Unable to reconnect now, perhaps due to a bug elsewhere. cc @mxsm.

关于您提的两个问题中的第一个,StreamObserver#onError()中调用#onCompleted()方法,一般不会导致无法重连。这种逻辑只是表示服务端出现问题导致流中断了,客户端直接处理为已接收响应。正常情况下,gPRC是默认支持重连的。现在无法重连,也许是其他地方的bug。抄送@mxsm

@pandaapo
Copy link
Member

pandaapo commented Nov 30, 2023

CC @mxsm

For the startup method of grpc springboot in examples, subStreamHandler is not created, so NPE will be thrown when resubscribe is called.

This point in your second question is a bug. When consumers use #subscribe (final List<subscriptionItem>subscriptionItems, final String URL) to subscribe under gPRC protocol, SubStreamHandler may become null.

关于您提的第二个问题中这一点,是个bug。当gPRC协议下,消费者订阅走的是#subscribe(final List<SubscriptionItem> subscriptionItems, final String url)SubStreamHandler就可能出现null的情况。

why org.apache.eventmesh.client.grpc.consumer.EventMeshGrpcConsumer#resubscribe is limited to using subStreamHandler for sending.

Regarding this point in your second question, wait a response from the community.

@mxsm mxsm self-assigned this Dec 12, 2023
@mxsm
Copy link
Member

mxsm commented Dec 12, 2023

@hhuang1231 @pandaapo I believe this is a bug. I will submit a PR to fix it.

@mxsm
Copy link
Member

mxsm commented Dec 13, 2023

this

spring boot application grpc problems I will submit next issue to fix it

@mxsm
Copy link
Member

mxsm commented Dec 15, 2023

This issue has been resolved through the corresponding pull request and will be closed.

@mxsm mxsm closed this as completed Dec 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

3 participants