Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix double tracking of memory allocated by LogCache and OperationTracker #1436

Closed
spolitov opened this issue May 25, 2019 · 0 comments
Closed

Fix double tracking of memory allocated by LogCache and OperationTracker #1436

spolitov opened this issue May 25, 2019 · 0 comments
Assignees
@spolitov
Labels
area/docdb YugabyteDB core features kind/bug This issue is a bug

Comments

@spolitov
Copy link
Contributor

LogCache and OperationTracker usually holds references to the same replicate messages.
But tracks them independently. That is causing this memory to be tracked twice.
@spolitov spolitov self-assigned this May 25, 2019
@spolitov spolitov added the kind/bug This issue is a bug label May 25, 2019
spolitov added a commit that referenced this issue Oct 9, 2019
@spolitov
[#1436] Fix operation memory double tracking
4c47115 
Summary:
Currently each Raft operation is registered in log cache and operation tracker.
Operation tracker would release operation right after it is applied.
Log cache would release operation after it is replicated to all nodes or log cache takes too many memory.
With one exception that log cache would not release operation that is referenced by someone else.

So not applied operation is tracked both by operation tracker and by log cache.
Also not applied operations could not be evicted from log cache.

This diff adds and uses mechanism to transfer mem tracking from operation tracker to log cache.
So unapplied operation memory would be tracked by mem tracker of operation tracker only.
After that it will be tracked by log cache mem tracker, while this operation present in log cache.

Also removed restrictions to evict log cache entries that are referenced by someone else.

This diff is also preparation work for fixing #2412.

Test Plan: Jenkins

Reviewers: mikhail, timur

Reviewed By: timur

Subscribers: ybase, bogdan

Differential Revision: https://phabricator.dev.yugabyte.com/D7317
@bmatican bmatican added the area/docdb YugabyteDB core features label Apr 21, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@spolitov spolitov

Labels
area/docdb YugabyteDB core features kind/bug This issue is a bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@bmatican @spolitov