I found myself needing a multithreaded multi part upload to S3 for some big files I was moving around. I found that only Fog had any existing support, and I didn't want yet another gem to install with my ServerTemplates so I decided to add the functionality to right_aws.
I wrote a few tests which do not fully cover my method, but I have used the method pretty extensively already and it's reasonably stable. It also appears as though the existing tests don't cover all of the existing code. Besides, I am a bit of a Ruby n00b and I couldn't get the entire suite of existing tests to pass, so I sorta gave up. ;-)
I posted in the RightScale forums about this as well; https://forums.rightscale.com/showthread.php?t=774
Hope this is useful for someone!
-Ryan J. Geyer-
Added Rightscale::S3Interface.store_object_multipart. Not a ton of te…
…sts, since I'm a bit of a Ruby n00b, and since the existing tests don't seem to fully cover the existing code
I just had an opportunity to try this out from a windows box and discovered that it doesn't exactly play nice there. Looks like I have some additional work to do for windows.
It worked great on my OSX box, and linux though. shrug
Your fix may not work correctly because right_aws is not thread-safe. We are looking into this issue now...
I was just wishing this existed and here it is! Thanks again Ryan.
No problem ryansch. As I'm sure you read above, this seems to work fine in *nix environments, but not so much in Windows.
I actually intend to refactor this code a bit such that right_aws is only providing the raw AWS method calls for S3 multi part. Then consuming applications can build their own multi-threaded approach to simultaneous uploading of chunks. I'll likely leave the existing store_object_multipart method in the file, but either comment it out, or move it up the chain to the S3 class which abstracts the S3Interface.
Hi guys, any news on this?