Support multiple send's with leavePartsOnError #634
Conversation
Track which parts have been uploaded and if we have an ETag for them, don't try to upload them again. This works well with `leavePartsOnError`, so that you can restart a failed upload and only upload the missing parts.
|
@lsegal thoughts? |
|
@AdityaManohar @lsegal bump? We've been using this in production for almost a month with success. |
| @@ -522,7 +532,7 @@ AWS.S3.ManagedUpload = AWS.util.inherit({ | |||
| */ | |||
| finishMultiPart: function finishMultiPart() { | |||
| var self = this; | |||
| var completeParams = { MultipartUpload: { Parts: self.completeInfo } }; | |||
| var completeParams = { MultipartUpload: { Parts: self.completeInfo.slice(1) } }; | |||
There was a problem hiding this comment.
@dconnolly Is there a reason that we are slicing out the first index here?
There was a problem hiding this comment.
We are slicing as we are following the pattern of keeping track of parts by the correctly indexed key, ie: part 1 has key value 1, and key 0 is null, so we slice it out. Somehow this worked without slicing (before this change) on the first upload pass, but any 'resumed' uploads get into a weird state when trying to push all the parts again.
|
@dconnolly I will look into adding some tests for this and then merge it in if everything looks good. |
|
That's great, thank you. |
|
+1(!!) Let's support pause / resume |
|
@dconnolly I've been using your version for some time now, works great! |
|
This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread. |
Track which parts have been uploaded and if we have an
ETagfor them, don't try to upload them again. This works well withleavePartsOnError, so that you can restart a failed upload and only upload the missing parts.This passes existing sdk tests and works well with buffer/file-type uploads. I don't think this is a use case for uploading streams, but it doesn't conflict with that case.
Fixes #633