Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UpdateConsensus could consume a lot of memory #1148

Closed
spolitov opened this issue Apr 5, 2019 · 0 comments
Closed

UpdateConsensus could consume a lot of memory #1148

spolitov opened this issue Apr 5, 2019 · 0 comments
Assignees
Labels
kind/bug This issue is a bug

Comments

@spolitov
Copy link
Contributor

spolitov commented Apr 5, 2019

During some workload UpdateConsensus could take a long time to proceed, but leader keep trying to send new requests each 3 seconds. All those requests waiting on update_lock_ taking a lot of memory.

@spolitov spolitov added the kind/bug This issue is a bug label Apr 5, 2019
@spolitov spolitov self-assigned this Apr 5, 2019
yugabyte-ci pushed a commit that referenced this issue Apr 6, 2019
Summary:
During some workload UpdateConsensus could take a long time to proceed, but leader keep trying to send new requests each 3 seconds.
All those requests waiting on update_lock_ taking a lot of memory.

Replaced std::mutex with std::timed_mutex and use RPC call timeout.
So only 1-2 pending updates would wait and consume memory.

Test Plan: ybd --gtest_filter QLStressTest.SlowUpdateConsensus

Reviewers: mikhail, bogdan

Reviewed By: bogdan

Subscribers: ybase, bharat

Differential Revision: https://phabricator.dev.yugabyte.com/D6444
@spolitov spolitov closed this as completed Apr 7, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug This issue is a bug
Projects
None yet
Development

No branches or pull requests

1 participant