Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Slow run when large pyz file #208

Open
sdementen opened this issue Feb 9, 2022 · 1 comment
Open

Slow run when large pyz file #208

sdementen opened this issue Feb 9, 2022 · 1 comment

Comments

@sdementen
Copy link

I have "shived" all my dependencies (pandas, scipy, ...) to end up with a compressed .pyz file of close to 200MB.
My entry point is a function that does not import anything but modules from the sdlib (sys, pathlib) and runs immediately when run outside the shiv file.
However, when I run the shiv file, it takes 16s to run (once it has already been decompressed in the .shiv folder, for the first run, it takes 82s to decompress to ~600 MB and run)
Is there some way to improve this performance ?
Is the .pyz file completely read from disk & uncompressed each time the pyz file is run (even if it is only to check it has already been decompressed in the .shiv folder) ?
If so, would it make sense to split the file in two file, a small .pyz with the bootstrap logic (check .shiv/... folder is updated, run preamble,...) and a large file with the zipped site-packages that would only be accessed if there is a need to decompress for the first time to the .shiv folder ?

@fquinner
Copy link

fquinner commented Apr 8, 2022

Thankfully my performance difference is a little smaller (0.6s vs 0.1s) but since it's a CLI utility which we execute a lot, it is certainly noticeable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants