-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
cmd: compact: clean partial / marked blocks concurrently #3115
cmd: compact: clean partial / marked blocks concurrently #3115
Commits on Sep 2, 2020
-
cmd: compact: clean partial / marked blocks concurrently
Clean partially uploaded and blocks marked for deletion concurrently with the whole compaction/downsampling process. One iteration could potentially take a few days so it should be nice to periodically clean unneeded blocks in the background. Without this, there are huge spikes in block storage usage. The spike's size depends on how long it takes to complete one iteration. The implementation of this is simple - factored out the deletion part into a separate function. It is called at the end of an iteration + concurrently if `--wait` has been specified. Add a mutex to protect from concurrent runs. Delete blocks from the deletion mark map so that we wouldn't try to delete same blocks twice or more. Signed-off-by: Giedrius Statkevičius <giedriuswork@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 2591e95 - Browse repository at this point
Copy the full SHA 2591e95View commit details
Commits on Sep 3, 2020
-
*: update changelog, e2e tests
Signed-off-by: Giedrius Statkevičius <giedriuswork@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 09d60b5 - Browse repository at this point
Copy the full SHA 09d60b5View commit details
Commits on Oct 16, 2020
-
Merge remote-tracking branch 'origin/master' into clean_partial_marke…
…d_periodically Signed-off-by: Giedrius Statkevičius <giedriuswork@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 8c14462 - Browse repository at this point
Copy the full SHA 8c14462View commit details -
cmd: compact: fix according to comments
Remove "error" from the `error` and just directly call the function. Signed-off-by: Giedrius Statkevičius <giedriuswork@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 2e26808 - Browse repository at this point
Copy the full SHA 2e26808View commit details
Commits on Oct 22, 2020
-
Merge remote-tracking branch 'origin/master' into clean_partial_marke…
…d_periodically Signed-off-by: Giedrius Statkevičius <giedriuswork@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 73c2b49 - Browse repository at this point
Copy the full SHA 73c2b49View commit details -
Forgot to remove this part while solving conflicts. Signed-off-by: Giedrius Statkevičius <giedriuswork@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for 8218d0c - Browse repository at this point
Copy the full SHA 8218d0cView commit details -
Signed-off-by: Giedrius Statkevičius <giedriuswork@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for d62aead - Browse repository at this point
Copy the full SHA d62aeadView commit details -
Signed-off-by: Giedrius Statkevičius <giedriuswork@gmail.com>
Configuration menu - View commit details
-
Copy full SHA for f97130f - Browse repository at this point
Copy the full SHA f97130fView commit details