Skip to content

Commit

Permalink
chunker: partially implement no-rename transactions (#4675)
Browse files Browse the repository at this point in the history
Some storage providers e.g. S3 don't have an efficient rename operation.
Before this change, when chunker finished an upload, the server-side copy
and delete operations that renamed temporary chunks to their final names
could take a significant amount of time.
This PR records transaction identifier (versioning) in the metadata of
chunker composite objects striving to remove the need for rename
operations on such backends.
This approach will be triggered be the new "transactions" configuration
option, which can be "rename" (the default) or "norename".
We implement the new approach for uploads (Put operations).
The chunker Move operation still uses the rename operation of
underlying backend. Filling this gap is left for a later PR.

Co-authored-by: Ivan Andreev <ivandeex@gmail.com>
  • Loading branch information
Maxwell Calman and ivandeex committed Feb 28, 2021
1 parent b029fb5 commit 9cc8ff4
Show file tree
Hide file tree
Showing 4 changed files with 317 additions and 53 deletions.
Loading

0 comments on commit 9cc8ff4

Please sign in to comment.