xds: implement xDS circuit breaking max_requests #7517
Conversation
…n the EDS lb policy.
| if (result.getStatus().isOk() && result.getSubchannel() != null) { | ||
| if (requestCount.get() >= maxConcurrentRequests) { |
There was a problem hiding this comment.
If result is a buffering result, should we drop the request immediately as well?
There was a problem hiding this comment.
result.getSubchannel() != null should be the thing to cover it.
There was a problem hiding this comment.
I think for buffering result (result.getStatus().isOk() && result.getSubchannel() == null), it should also be dropped immediately.
There was a problem hiding this comment.
Why? It will cause RPCs to be dropped prematurely. We shouldn't do anything to buffer result. The channel will handle it correctly.
There was a problem hiding this comment.
If requestCount.get() >= maxConcurrentRequest it should drop the RPC regardless of the subchannel's readiness. What's dropped prematurely?
There was a problem hiding this comment.
No, that should not count toward the number of dropped requests. Also, it doesn't make sense to override the non-ready picker with a drop picker.
|
@dapengzhang0 Ping. |
Implemented xDS circuit breaking for the maximum number of requests can be in-flight. The threshold is retrieved from CDS responses and is configured at the cluster level. It is implemented by wrapping the Picker spawned by EDS LB policy (which resolves endpoints for a single cluster) with stream-limiting logic. That is, when the picker is trying to create a new stream (aka, a new call), it is controlled by the number of open streams created by the current EDS LB policy. RPCs dropped by circuit breakers are recorded into total number of drops at cluster level and will be reported to TD via LRS. In the future, multiple gRPC channels can be load balancing requests to the same (global) cluster. Those request should share the same quota for maximum number of requests can be in-flight. We will use a global counter for aggregating the number of currently-in-flight requests per cluster.
No description provided.