[rel-v0.24] Enhancement: Implement Chunk Deletion for Multi-part Uploaded Files on GCP and Openstack Cloud Providers #682
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…n GCP and Openstack Cloud Providers (gardener#673) * Added generic functionality to garbage collect the chunks when GC is triggered instead of waiting for the chunks to get deleted when the corresponding snapshot is garbage collected * Deleting chunks from GCS bucket immediately after the composite object is uploaded * Deleting chunks from OpenStack bucket immediately after the composite object is uploaded * Add logs for count of chunks getting deleted * Updated comments and code for readability * Fixed a bug in tests * Transfer the immediate chunk deletion logic to new function with context timeout * Timeout same as chunk upload timeout * Collected individual chunk deletion errors into a errList * Affected code: Gcs & Swift snapstore chunk deletion * Add logs after successfully performing chunk deletion * Fix snapstore tests, update comments * Parallelize chunk deletion * For better performance * To prevent blocking the snapshotter while chunk deletion for older snapshot is underway * Add comment * Add mutex to prevent DATA RACE in ObjectMap of mock GCS & Swift clients while deleting chunks in parallel * Minor fixes * Add mutex to prevent parallel read & write for GCS Mock client * Improved error collection in chunk deletion * Remove immediate chunk deletion, address edge case in Garbage Collection * Address review comments * Minor fixes
gardener-robot
added
needs/review
gardener-robot-ci-3
added
reviewed/ok-to-test
gardener-robot
added
the
status/closed
shreyas-s-rao
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
Cherrypick of PR: #673
Which issue(s) this PR fixes:
Fixes #
Special notes for your reviewer:
Release note: