Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OpenBLAS package slimming #642

Closed
jaakkor2 opened this issue Mar 23, 2020 · 5 comments
Closed

OpenBLAS package slimming #642

jaakkor2 opened this issue Mar 23, 2020 · 5 comments

Comments

@jaakkor2
Copy link
Contributor

Downloading big jll packages can be problematic with unreliable network connections as described in https://discourse.julialang.org/t/install-binary-package-such-as-openblas-jll-via-very-slow-internet-connection . One way to improve the situation could be trimming the size of the package.

OpenBLAS_jll could be slimmed by (I am referring to the Windows version)

  • removing static libs (*.a) files in lib-directory 84 MB
  • removing logs/OpenBLAS.log 5.8 MB

OpenBLAS.v0.3.7.x86_64-w64-mingw32-libgfortran5.tar.gz would go from 27 MB to 8 MB proving a speedup for downloading, and disk space would go s from 123.7 MB to 33.6 MB.

@giordano
Copy link
Member

The log files are very important. Also, they're plain text, usually they can be compressed pretty well. Static archives are less important instead6

@davidanthoff
Copy link
Contributor

If some of this stuff is important to have for the folks that build these packages, couldn't there be to archives created, one that includes things like log files, and a second one that doesn't that is actually deployed to user systems that really only has what is needed?

@haampie
Copy link
Contributor

haampie commented Mar 24, 2020

I don't really understand where you get those numbers @jaakkor2? E.g. https://github.com/JuliaBinaryWrappers/OpenBLAS_jll.jl/releases has

 OpenBLAS.v0.3.7.x86_64-w64-mingw32-libgfortran3.tar.gz
26.4 MB
OpenBLAS.v0.3.7.x86_64-w64-mingw32-libgfortran4.tar.gz
26.6 MB
OpenBLAS.v0.3.7.x86_64-w64-mingw32-libgfortran5.tar.gz
26.4 MB

Edit, sorry, I just looked at the uncompressed number of 84MB, nevermind.


I also tried stripping the shared libraries, that saves just a hair:

lib $ cp libopenblas64_.0.3.7.so libopenblas64_.0.3.7.so_
lib $ strip --strip-all ./libopenblas64_.0.3.7.so
lib $ du -sh libopenblas64_.0.3.7.so
29M	libopenblas64_.0.3.7.so
lib $ du -sh libopenblas64_.0.3.7.so_ 
30M	libopenblas64_.0.3.7.so_

maybe not worth it.

@haampie
Copy link
Contributor

haampie commented Mar 24, 2020

With #621's change locally on x86_64 + libgfortran.so.4 I get

lib $ du -sh ../../openblas_tmp.tar.gz 
5.2M	../../openblas_tmp.tar.gz

😌

@jaakkor2
Copy link
Contributor Author

Fixed by #621. Thanks!

OpenBLAS.v0.3.9.x86_64-w64-mingw32-libgfortran5.tar.gz is 8.84 MB, about 3x improvement.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants