Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix copying of files > 5GB via aws-fog. #2583

Merged
merged 1 commit into from Aug 26, 2021

Conversation

slonopotamus
Copy link
Contributor

@slonopotamus slonopotamus commented Jul 7, 2021

This change utilizes multipart-upload logic from Fog::AWS::Storage::File#copy that was added in
fog/fog-aws@e3a9a35

This change utilizes multipart-upload logic from Fog::AWS::Storage::File#copy that was  added in
fog/fog-aws@e3a9a35
@jwstric2
Copy link

This would be a great addition, we were just pondering if the support for multipart uploads (were fixing to test 5 gig+ uploads) was there in carrierwave/fog integration

@mshibuya mshibuya merged commit 27218a9 into carrierwaveuploader:master Aug 26, 2021
@mshibuya
Copy link
Member

I would say that there's a design fault in fog-aws (similar methods Storage#copy_object and File#copy exist, but they behave slightly differently!), but working around this in our side sounds reasonable. I understand the pain of not being able to upload files more than 5GB size. It is common for the video files to exceed 5GB.

Thanks for the PR.

@slonopotamus
Copy link
Contributor Author

slonopotamus commented Aug 26, 2021

As I understand it, Storage#copy_object (or maybe even all Storage#xxx methods) are kind of low-level API that directly maps to S3 calls. On the other side, File#xxx are higher-level functions that encapsulate underlying API calls in nontrivial ways.

@slonopotamus slonopotamus deleted the 5GB-copy branch August 26, 2021 08:51
mshibuya added a commit that referenced this pull request Aug 26, 2021
@rsalehi2013
Copy link

@mshibuya any chance we can get a new release that includes this patch? thank you

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants