Multipart upload

[domenic]

Consider adding support for the multipart upload API. See #17 and #50 for stale pull requests.

We still need to determine what the API would be, how much it would change or add to Knox, and whether it's in scope for Knox or should perhaps be a different package.

[domenic]

See also #92 and this StackOverflow explanation of multipart upload.

[joehoyle]

+1, after trying to upload a 15GB file, I stumbled on this issue :)

Is there anyway I can work around this / hack something together with knox?

[domenic]

I mean, you can use client.request() to send raw requests of any kind to S3, and using that you could fulfill the algorithm outlined in that StackOverflow answer.

And then, of course, you could make sure that code was sufficiently general, and submit it as a pull request to Knox! :)

[lazdmx]

+1 vote

[nathanoehlman]

I've written a standalone module (knox-mpu) that uses knox to provide multipart uploads to S3. It's got a few features missing at the moment, but does the job.

If you guys are interested, I'll add it to knox and submit a pull request.

[domenic]

@nathanoehlman Oh awesome! I'll check it out soon. I'd lean toward keeping it a separate package and linking to it from the readme, but maybe there'd be value in transparently switching to MPU for certain cases.

[nathanoehlman]

@domenic Yep - those were my thoughts exactly. I originally had it in Knox but it didn't feel like it quite belonged (hence the separate module) so I figured I'd just leave the decision as to where it belonged to you guys :)