Abort failed multipart uploads on RuntimeException #786
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
If a file > 5MB is being uploaded to a bucket, this client will initiate a multipart upload. However, if in the course of uploading that file a RuntimeException occurs, this library will not abort the multipart upload. This is especially possible when streaming data across a network connection into a bucket. This was introduced in minio#741 when resuming failed multipart uploads was removed. Reproduction steps: * Use this script: https://gist.github.com/Dorthu/b75f2a4ad9e0e453a040412438d3fe32 * Replace the server URL and port, access key, secret key, and bucket name with valid values * Compile/run the script * Observe the state of multipart uploads in the bucket after running with `s3cmd multipart s3://bucket-name` (or another tool) Subsequent runs of the script create new multipart uploads in the bucket. I specifically noticed this bug when using this library against Ceph, if that's relevant.
balamurugana
reviewed
Aug 6, 2019
balamurugana
approved these changes
Aug 6, 2019
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
If a file > 5MB is being uploaded to a bucket, this client will initiate
a multipart upload. However, if in the course of uploading that file a
RuntimeException occurs, this library will not abort the multipart
upload. This is especially possible when streaming data across a
network connection into a bucket. This was introduced in #741 when
resuming failed multipart uploads was removed.
Reproduction steps:
valid values
with
s3cmd multipart s3://bucket-name(or another tool)Subsequent runs of the script create new multipart uploads in the
bucket.
I specifically noticed this bug when using this library against Ceph, if
that's relevant.