Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Old deletes need to be purged during Compaction #112

Open
weggert opened this issue Mar 4, 2015 · 1 comment
Open

Old deletes need to be purged during Compaction #112

weggert opened this issue Mar 4, 2015 · 1 comment

Comments

@weggert
Copy link
Collaborator

weggert commented Mar 4, 2015

Before 'live' compaction, DELETE events older than 7 days (by default) were compacted out during WalTool compaction.

When Sirius switched to using live compaction, that bit of logic was left out. So if a DELETE event is not followed by a subsequent PUT then it lives in the Uberstore forever.

For our usage, this has led to more than 165,000,000 stale DELETE events in the Uberstore. That represents almost 3x the live PUTs and 15gb out of the 40gb Uberstore.

Sirius needs to re-add the ability to purge DELETE events that are older than a specified age while doing compaction.

@clinedome-work
Copy link
Collaborator

Thanks for the issue and the immediate PR. Reviewing now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants