-
Notifications
You must be signed in to change notification settings - Fork 989
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Replace StreamBall, replace tarballs with zip #10919
Replace StreamBall, replace tarballs with zip #10919
Conversation
I think we could still take into account the value of |
The way StreamBall writes to the wsgi response just doesn't work for ASGI frameworks. The zipstream method seems superior in that it creates a uncompressed zip archive and doesn't need a tempfile. Additionally I think it makes sense to return zip files, as that is probably the most well known compression format, and we could replace this conditionally with https://www.nginx.com/resources/wiki/modules/zip/ (while we also work on creating archives in a celery worker). zipstream was one of the suggested libraries in galaxyproject#9794 (comment)
95f1352
to
d122df8
Compare
Alright, I've implemented returning a header that is compatible with mod-zip. admins (@natefoo @gmauro @hexylena), do you think https://github.com/galaxyproject/galaxy/blob/d122df8e9b9fb9b6c5a32c54d132487fa44cda30/doc/source/admin/nginx.md#creating-archives-with-mod-zip is reasonable ? An alternative we could implement (but that is more work, and I'm not sure time wouldn't be better spent completely offloading archive creation to celery or jobs) would be to return locations to |
e644a72
to
3af98cd
Compare
3af98cd
to
d4552d3
Compare
Co-authored-by: Nicola Soranzo <nicola.soranzo@gmail.com>
Co-authored-by: Nicola Soranzo <nicola.soranzo@gmail.com>
FWIW, although the option specifically mentions |
The |
} | ||
``` | ||
|
||
The `internal;` statement means that the location can only be used for internal nginx requests. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
❤️
The way StreamBall writes to the wsgi response just doesn't work for ASGI frameworks.
The zipstream method seems superior in that it creates a uncompressed zip archive and doesn't need a tempfile.
Additionally I think it makes sense to return zip files, as that is probably the most well known compression format.
This also adds the option to have nginx serve zip archives directly using mod_zip