Need for circuit breaker integration in ETCD client library

[shivam0891]

Hi etcd-folks,
We have fleet size of 1500+ ETCD clusters and we have seen lot of issues due to 1 degraded node in ETCD(be it slow disk, stale follower, noisy neighbour, etcd in built rate limiter) due to lack of circuit breaker in ETCD client.
We have build the wrapper client and provided the circuit breaker to solve these issues and so far it is working fine. So i wanted to check the feasibility to add this capability to etcd client itself? so you see any concern in this?

[serathius]

From community meeting, for me seems related to https://github.com/kubernetes/enhancements/pull/6319/changes

[ahrtr]

If we accept this, we should ensure it's more generic instead of a bespoken solution. Also if we support #22328, it should also resolve this case, but the concern on that one is that is uses grpc's experimental package.

[ahrtr]

We are not challenging the issue itself, but more about the solution.

[gyuho]

Would love to learn more about how this is done (e.g., what threads/time windows do you use for circuit breaker), and how this can be done more natively, using existing grpc-go primitives