Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

kvcoord: improve DistSender circuit breaker scalability #119917

Closed
erikgrinaker opened this issue Mar 5, 2024 · 0 comments · Fixed by #119865
Closed

kvcoord: improve DistSender circuit breaker scalability #119917

erikgrinaker opened this issue Mar 5, 2024 · 0 comments · Fixed by #119865
Assignees
@erikgrinaker
Labels
A-kv-client Relating to the KV client and the KV interface. C-performance Perf of queries or internals. Solution not expected to change functional behavior. T-kv KV Team
Projects
KV

Comments

@erikgrinaker
Copy link
Contributor

erikgrinaker commented Mar 5, 2024 •
edited by exalate-issue-sync bot

The DistSender circuit breakers added in #118943 don't scale well. In particular, they will launch a probe goroutine for each replica that keeps running for the duration that the circuit breaker is tripped. We should improve this, e.g. by only probing replicas that are seeing (rejected) user traffic.

Jira issue: CRDB-36386

Epic CRDB-25200
@erikgrinaker erikgrinaker added C-performance Perf of queries or internals. Solution not expected to change functional behavior. A-kv-client Relating to the KV client and the KV interface. T-kv KV Team labels Mar 5, 2024
@erikgrinaker erikgrinaker self-assigned this Mar 5, 2024
@blathers-crl blathers-crl bot added this to Incoming in KV Mar 5, 2024
@erikgrinaker erikgrinaker mentioned this issue Mar 5, 2024
Merged
@craig craig bot closed this as completed in f1257ab Mar 12, 2024
KV automation moved this from Incoming to Closed Mar 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@erikgrinaker erikgrinaker

Labels
A-kv-client Relating to the KV client and the KV interface. C-performance Perf of queries or internals. Solution not expected to change functional behavior. T-kv KV Team
Projects
KV
Closed
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

kvcoord: don't probe idle replicas with tripped circuit breakers erikgrinaker/cockroach
1 participant
@erikgrinaker