Binary builds from RSPM blow up image size

[benz0li]

Reason: The dynamic libraries are not stripped.

rocker/tidyverse (Ubuntu 20.04, RSPM, binary install):

root@89f0a27ec269:/usr/local/lib/R/site-library/vroom/libs# du -h vroom.so 
24M	vroom.so
root@89f0a27ec269:/usr/local/lib/R/site-library/vroom/libs# file vroom.so
vroom.so: ELF 64-bit LSB shared object, x86-64, version 1 (GNU/Linux), dynamically linked, BuildID[sha1]=88a2384029c2331d1d8f91274db23fb30edb9668, with debug_info, not stripped

custom build (Debian 11, CRAN, source install):

root@bfc0aad5ae64:/usr/local/lib/R/site-library/vroom/libs# du -h vroom.so 
848K	vroom.so
root@bfc0aad5ae64:/usr/local/lib/R/site-library/vroom/libs# file vroom.so
vroom.so: ELF 64-bit LSB shared object, x86-64, version 1 (GNU/Linux), dynamically linked, BuildID[sha1]=f6d17acb2cf4f8e75af5e97dcb7a17297ff5f9ee, stripped

Total size of /usr/local/lib/R/site-library:

• rocker/tidyverse: 732 MB
• custom build: 198 MB

@eddelbuettel Can you think of a reason why the dynamic libraries of the RSPM binaries are not stripped?

[eitsupi]

FYI, since rocker/tidyverse:devel has source installations, we can see this in the comparison between rocker/tidyverse:latest and rocker/tidyverse:devel.

[eddelbuettel]

@benz0li No idea. I am not all that involved with rspm and use it little. You could try to strip post-installation... Reason may be a simple -g default for system-wide compiler settings.

[benz0li]

@eddelbuettel Thanks for your feedback. Stripping the dynamic libraries post-installation reduces the size of /usr/local/lib/R/site-library as expected.

rocker/tidyverse (Ubuntu 20.04, RSPM, binary install):

root@769bd0aff019:/# du -sh /usr/local/lib/R/site-library/
732M	/usr/local/lib/R/site-library/
root@769bd0aff019:/# find /usr/local/lib/R/site-library/*/libs/ -name \*.so | xargs strip -s -p
root@769bd0aff019:/# du -sh /usr/local/lib/R/site-library/
198M	/usr/local/lib/R/site-library/

@eitsupi If you include stripping [the dynamic libraries] in your Dockerfiles/installation-scripts, it would reduce the image size significantly.

[benz0li]

I will contact RStudio and ask if there is a particular reason why the dynamic libraries of the RSPM binaries are not stripped.

[cboettig]

🎉 Thanks for the report, the suggestion to strip the shared libs on the images here and the follow up with RStudio :-)

[benz0li]

@jjallaire made sure this information gets to the right person.

[bdeitte]

I'm not sure on the reason here, and I'm checking with the team. You're also welcome to tag myself or @tylfin in the future and we can try to answer public package manager questions that come up.

[eddelbuettel]

It may be a side-effect of how 'default' compilation on Linux is setup. Debian, for example, always builds with -g ("devs want debug symbols") but then puts them into optional -dbg packages you can load on demand (which is actually slick). But if then do no strip the symbols you end up with bloat :-/ I blogged about that a few times:

• r^4 #9 Compacting your Shared Libraries
• r^4 #10 Compacting your Shared Libraries, After The Build
• r^4 #21 A Third and Final (?) Post on Stripping R Libraries

but lets just say that I didn't manage to get R CMD INSTALL changed to use stripping by default 😿

So my suggestion above to do it locally (after RSPMs are unpacked) may be best for Rocker users; you guys could and should look into stripping at source. It will make things smaller and hence faster to ship across the wire and install.

[bdeitte]

This has started an interesting conversation here, so thanks for bringing this up. The thoughts here are that this wasn't originally a conscious choice, that we just run R's package install command with mostly default flags/options. @glin found the commend about --strip interesting but also noted the R extensions manual about CRAN prohibiting the stripping of debug symbols:

1.6 Writing portable packages
The strip utility is platform-specific (and CRAN prohibits removing debug symbols). For example the options --strip-debug and --strip-unneeded of the GNU version are not supported on macOS nor Solaris83: the POSIX standard for strip does not mention any options, and what calling it without options does is platform-dependent. Stripping a .so file could even prevent it being dynamically loaded into R on an untested platform.

Our suggestion right now would be to have you strip this as part of building your images, but we're still discussing it.

[benz0li]

Thanks for all the additional information. Very interesting, indeed.

The following has no effect on package installations from RSPM when building the rocker images:

rocker-versioned2/scripts/bin/install2.r

Lines 73 to 74 in ea351bc

	## ensure installation is stripped
	Sys.setenv("_R_SHLIB_STRIP_"="true")

Regarding 1.6 Writing portable packages: As RSPM serves pre-compiled binaries that are specific per Linux distribution + version, is there any harm in stripping the .so files then?

[eddelbuettel]

The following has no effect on package installations

Yes. It only would if you compiled and linked so that a strip step could be added at linking. But by using RSPM you are relying on pre-made binaries by design so no extra link step. You would need something like (riffing here, untested, but I have done this ...) strip /usr/local/lib/R/site-library/*/libs/*.so (and same for /usr/lib/....).

Maybe at this point you could test the is there any harm question via a locally-modified container? There shouldn't ...

[benz0li]

Not actively using rocker images myself as I have moved to Jupyter (JupyterHub with JupyterLab + code-server) for some time now.
ℹ️ Multi-arch (linux/amd64, linux/arm64/v8) Debian-based images with stripped source-installations of Git, R and add-on packages from CRAN — no problems so far.

[benz0li]

Closing due to inactivity.

[eitsupi]

I was hoping to hear from @bdeitte about his future plans rather than strip every installation in this repository now.