CRAN version of readr crashes the compiler on small-RAM Ubuntu 16.04 system

[znmeb]

I'm building a system on a Ubuntu 16.04LTS base. On a 1024 MB (64-bit) virtual machine I get this:

> install.packages("readr")
Installing package into ‘/home/ubuntu/R/x86_64-pc-linux-gnu-library/3.3’
(as ‘lib’ is unspecified)
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  191k  100  191k    0     0   505k      0 --:--:-- --:--:-- --:--:--  505k
* installing *source* package ‘readr’ ...
** package ‘readr’ successfully unpacked and MD5 sums checked
** libs
g++ -I/usr/share/R/include -DNDEBUG   -I"/home/ubuntu/R/x86_64-pc-linux-gnu-library/3.3/Rcpp/include" -I"/home/ubuntu/R/x86_64-pc-linux-gnu-library/3.3/BH/include"   -fpic  -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g  -c Collector.cpp -o Collector.o
virtual memory exhausted: Cannot allocate memory
/usr/lib/R/etc/Makeconf:141: recipe for target 'Collector.o' failed
make: *** [Collector.o] Error 1
ERROR: compilation failed for package ‘readr’
* removing ‘/home/ubuntu/R/x86_64-pc-linux-gnu-library/3.3/readr’
* restoring previous ‘/home/ubuntu/R/x86_64-pc-linux-gnu-library/3.3/readr’

The downloaded source packages are in
    ‘/tmp/RtmpnY2dFK/downloaded_packages’
Warning message:
In install.packages("readr") :
  installation of package ‘readr’ had non-zero exit status
> 

If I push the RAM up to 1536 MB it compiles. But if I enable two-job "make" it crashes again.

$ gcc --version
gcc (Ubuntu 5.4.0-6ubuntu1~16.04.2) 5.4.0 20160609

This is in a Vagrant box, so I can try various experiments.

[iliastsergoulas]

I seem to have a similar problem (described here http://stackoverflow.com/questions/40945512/r-ubuntu-cant-install-packages-readr-and-eurostat). Is memory the issue here?

[znmeb]

@iliastsergoulas It was for me - I raised the RAM and it compiled. How much RAM do you have?

[iliastsergoulas]

@znmeb I got a virtual server with 1 GB RAM. I know, I got the cheapest because I did not have high demands from this server.

[znmeb]

@iliastsergoulas Yeah - 1 GB isn't enough to compile that routine. You might be able to get a per-compiled binary for readr at R-Hub ... https://builder.r-hub.io/

I'm going to start testing my build scripts with R-Hub next week; I have a bunch of Docker images with header files required to compile packages.

[eddelbuettel]

This StackOverflow question referenced this issue. I just provided a complete solution over there which I may as well quote here:

You have insufficient RAM on that machine. So you have two options:

• continue what you are doing and trying to compile from source, which will need more memory and is likely to fail
• install a prebuilt binary (!!) so that you do not have to compile in the first place

The easiest way is something like this (and I am showing only the commands, not the output while I do
this in a Docker instance of Ubuntu 16.04, where the account is already root)

apt-get update     # refresh
apt-get install software-properties-common
add-apt-repository -y "ppa:marutter/rrutter"
add-apt-repository -y "ppa:marutter/c2d4u"
apt-get update     # now with new repos
apt-get install r-cran-readr

and voila you have the readr package. Now proceed for eurostat which is not packaged as a binary,
but may not require heavier resources which readr does due to its C++ code.

What is suggested here (ie using r-hub) is a waste of resources as we already have (most of) these binaries. Michael's PPAs cover over 3000 packages for several Ubuntu flavours.

[iliastsergoulas]

Yes, that worked for me fine! Thank you for your solution @eddelbuettel

[znmeb]

@eddelbuettel I just discovered Michael's PPA thanks to your comment here - how come there's no link to it on https://cran.rstudio.com/bin/linux/ubuntu/? The PPA page itself implies there is one - https://launchpad.net/~marutter/+archive/ubuntu/rrutter

[eddelbuettel]

Those PPAs are neither hidden nor secret nor new.

Michael had them for over a decade, methinks, and they have been mentioned on r-sig-debian and other places. A quick search suggest 57,400 Google hits for "'michael rutter' c2d4u'".