-
-
Notifications
You must be signed in to change notification settings - Fork 4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Question]How to implement grpc connection pool via kratos 2.x #2860
Comments
gRPC is based on HTTP2 and has the feature of multiplexing. It does not need to use Connection pool |
Thanks for the fast response. I'm having one issue with my current app. I have a benchmarking which would fire 10k go routines with a simple grpc request. I noticed that 10k is max number of go-routines I can fire without facing "grpc connection timeout" or "grpc retry xxx" issue. I used a single grpc client though(balancer is kratos's WRR one). May I know what are the possible ways to make my service able to handle more than 10k concurrent requests without error? Buffered channel as semaphore is a possible approach but I prefer to fix it in either system level or in kratos level. Tests are done in aliyun k8s cluster I'm referring these article too: |
Hi, @fans3210! I'm Dosu, and I'm here to help the kratos team manage their backlog. I wanted to let you know that we are marking this issue as stale. Based on my understanding of the issue, you were seeking help with implementing a gRPC connection pool using Kratos 2.x. You mentioned that you couldn't find materials on this topic in the newer versions of Kratos. In the comments, another user suggested that gRPC doesn't need a connection pool due to its multiplexing feature. However, you explained that you are facing issues with your current app and want to handle more than 10k concurrent requests without errors. You also provided links to articles you are referring to. Before we proceed, we would like to confirm if this issue is still relevant to the latest version of the kratos repository. If it is, please let us know by commenting on this issue. Otherwise, feel free to close the issue yourself, or the issue will be automatically closed in 7 days. Thank you for your understanding, and we look forward to hearing from you soon. |
Hi @dosubot. The issue is not resolved yet. Would be appreciated if kratos team can provide more details with explanations on some best practices of implementing grpc pool to achieve higher qps |
@shenqidebaozi Could you please help @fans3210 with their issue regarding implementing a gRPC connection pool in Kratos 2.x? They are seeking best practices to achieve higher QPS and would appreciate more details and explanations. Thank you! |
Hi, @fans3210, I'm helping the Kratos team manage their backlog and am marking this issue as stale. It seems like you are seeking guidance on implementing a gRPC connection pool using Kratos 2.x, and there was a discussion about the necessity of a connection pool for gRPC due to its multiplexing feature. You mentioned facing issues with handling more than 10k concurrent requests without errors and requested details and explanations on best practices for implementing a gRPC pool to achieve higher QPS. Could you please confirm if this issue is still relevant to the latest version of the Kratos repository? If it is, please let the Kratos team know by commenting on the issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days. Thank you for your understanding and cooperation. |
Hi @dosubot. The issue is not resolved yet. Would be appreciated if kratos team can provide more details with explanations on some best practices of implementing grpc pool to achieve higher qps |
in grpc doc
and in grpc github, issue 21386 still not resolved. |
To address the challenge of handling more than 10k concurrent gRPC requests in Kratos 2.x, consider implementing a client connection pool and leveraging a custom load balancer. Here's a strategy that aligns with Kratos' capabilities and the nuances of gRPC's performance characteristics:
Here's an example of how you might configure a gRPC client with these optimizations in Kratos: ctx := context.Background()
conn, err := grpc.DialInsecure(ctx,
WithEndpoint("your_service_endpoint"),
WithDiscovery(yourDiscoveryImplementation),
WithBalancerName("selector"), // Custom balancer
WithTimeout(2000*time.Millisecond),
WithMiddleware(yourMiddleware...),
WithUnaryInterceptor(yourUnaryInterceptors...),
WithStreamInterceptor(yourStreamInterceptors...),
WithOptions(yourDialOptions...), // Custom gRPC dial options
)
if err != nil {
log.Fatalf("did not connect: %v", err)
}
defer conn.Close() This setup, leveraging Kratos' features and custom balancer, aims to efficiently manage connections and distribute load, helping to mitigate issues related to handling high numbers of concurrent requests.
|
Please see the FAQ in our main README.md before submitting your issue.
In older version of kratos, there was a grpc connection pool implementation. But I'm not able to find such materials in 2.x versions. Appreciate if could help give some examples or introductions.
The text was updated successfully, but these errors were encountered: