Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stream into stream #80

Closed
1oglop1 opened this issue Mar 3, 2022 · 1 comment
Closed

Stream into stream #80

1oglop1 opened this issue Mar 3, 2022 · 1 comment

Comments

@1oglop1
Copy link

1oglop1 commented Mar 3, 2022

Hi, thank you for the inspirational code.

I'm wondering if it's possible to adapt your code and make it compatible with S3 storage.

I use django-storages and boto3 client returns a streaming body (already open fp) and all metadata.

I need a zipfile (or any other archive) to be created from the set of files during the GET request (not great but it has to be).

So to save the memory I have 2 options.

  1. Use zipfly as is and download files into a temporary location (and remove it after the operation)
  2. Or better solution that doesn't require intermediate storage so I could pipe the content of streaming body into zipfly and return that as a streaming response.

streaming body (many of them) -> zipfly(zip file) -> streaming response

Do you think that option two is possible?
If so, could you please point out what pieces of code I should focus on to adapt zipfly?

Thank you

@sandes
Copy link
Owner

sandes commented Mar 3, 2022

Hello :)
Mount your S3 bucket via FUSE using s3fs-fuse and stream the file using django

@sandes sandes closed this as completed Mar 3, 2022
@sandes sandes pinned this issue Mar 3, 2022
@sandes sandes reopened this Oct 20, 2022
@sandes sandes closed this as completed Oct 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants