Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[WIP] Enforce object store heap memory constraints during recovery #298

Closed
wants to merge 25 commits into from
Closed

[WIP] Enforce object store heap memory constraints during recovery #298

wants to merge 25 commits into from

Conversation

yrajas
Copy link
Contributor

@yrajas yrajas commented Apr 22, 2024

During recovery,

  • Tracks heap memory usage by observing page deserializing operations.
  • If heap memory constraint is to be enforced, pages are read in one at a time.
  • If memory limit is violated, pages are freed until memory is brought within the limit again.

Yoganand Rajasekaran added 25 commits April 18, 2024 16:31
Initial draft.
020dbba
Additiional changes.
06a5b0a
Fixed setting head address.
04fa771
Cleanup.
f59de06
Updates to free previously allocated pages that could violate size co…
21e14cc 
…nstraint.

Added a wait for page flush before reading into the same slot.
Moved to a batch read approach.
62eaf26
Added logger to test instance of GarnetServer.
497a1e7
Cleanup.
4d5fbb7
Fix read page range up to snapshotendpage.
7c32494
Resolved PR comments.
c6c2d73
Added benchmark.
d555a92 
Resolved PR comments.
@yrajas
Initial draft.
21e6a9f
@yrajas
Additiional changes.
808f879
@yrajas
Fixed setting head address.
4881044
@yrajas
Cleanup.
ae4a213
@yrajas
Updates to free previously allocated pages that could violate size co…
48b986a 
…nstraint.

Added a wait for page flush before reading into the same slot.
@yrajas
Moved to a batch read approach.
7fd86b8
@yrajas
Added logger to test instance of GarnetServer.
ed14a56
@yrajas
Cleanup.
f3d88d6
@yrajas
Fix read page range up to snapshotendpage.
079f00d
@yrajas
Resolved PR comments.
bc39938
@yrajas
Added benchmark.
88ceb33 
Resolved PR comments.
Merge branch 'yrajas/recoverysizecheck' of https://github.com/yrajas/…
b42d64f 
…garnet into yrajas/recoverysizecheck
Fixed formatting.
46968c9
Initial commit.
bf4e3d3
@yrajas yrajas closed this May 5, 2024
@yrajas yrajas deleted the yrajas/objstorerecoverysize branch June 21, 2024 18:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
1 participant
@yrajas