R and Rdevel share some libpaths

[wch]

Here are the libpaths for each:

$ docker run rocker/r-devel Rscript -e ".libPaths()"
[1] "/usr/local/lib/R/site-library" "/usr/lib/R/site-library"      
[3] "/usr/lib/R/library"           

$ docker run rocker/r-devel Rscriptdevel -e ".libPaths()"
[1] "/usr/local/lib/R/site-library" "/usr/local/lib/R/library"     
[3] "/usr/lib/R/library"           

This causes problems when you install packages with Rdevel and then load them with R (and possibly vice versa).

$ docker run --rm -ti rocker/r-devel /bin/bash

# Rscriptdevel -e "install.packages('bitops', repos='http://cran.rstudio.com')"

root@f99ce81d4c09:/# Rscript -e "library(bitops)"
Warning message:
package ‘bitops’ was built under R version 3.2.0 

root@f99ce81d4c09:/# Rscript -e "packageDescription('bitops')"
Package: bitops
Version: 1.0-6
Date: 2013-08-17
Author: S original by Steve Dutky <sdutky@terpalum.umd.edu> initial R
        port and extensions by Martin Maechler; revised and modified by
        Steve Dutky
Maintainer: Martin Maechler <maechler@stat.math.ethz.ch>
Title: Bitwise Operations
Description: Functions for bitwise operations on integer vectors.
License: GPL (>= 2)
Packaged: 2013-08-17 15:58:57 UTC; maechler
NeedsCompilation: yes
Repository: CRAN
Date/Publication: 2013-08-17 21:10:34
Built: R 3.2.0; x86_64-unknown-linux-gnu; 2014-10-24 00:29:31 UTC; unix

-- File: /usr/local/lib/R/site-library/bitops/Meta/package.rds 

There are a total of 4 different directories. After running the commands above, here are the contents of each:

root@f99ce81d4c09:/# ls /usr/lib/R/site-library/
root@f99ce81d4c09:/# ls /usr/local/lib/R/site-library/
bitops  docopt  stringr
root@f99ce81d4c09:/# ls /usr/lib/R/library/
base   cluster    datasets  grDevices   lattice  methods  nnet      spatial  stats4    tools
boot   codetools  foreign   grid    MASS     mgcv     parallel  splines  survival  translations
class  compiler   graphics  KernSmooth  Matrix   nlme     rpart     stats    tcltk     utils
root@f99ce81d4c09:/# ls /usr/local/lib/R/library/
base  compiler  datasets  graphics  grDevices  grid  methods  parallel  splines  stats  stats4  tcltk  tools  translations  utils

Here's what it looks like each one of these is used for:

• /usr/local/lib/R/site-library/: Packages installed by root in both R and Rdevel.
• /usr/lib/R/site-library/: R has this in its libpath, but nothing seems to be there. This seems to serve no purpose, though I could be wrong.
• /usr/lib/R/library/: base packages for R, and packages installed via .debs, like r-cran-mass. This directory is also in Rdevel's libpath.
• /usr/local/lib/R/library/: base packages for Rdevel.

I assume that the libpaths for R are OK, since it seems to be the system default. Then the problems for Rdevel's libpath, c("/usr/local/lib/R/site-library", "/usr/local/lib/R/library", "/usr/lib/R/library"), are:

• /usr/local/lib/R/site-library is shared with R.
• /usr/lib/R/library contains the base packages for R 3.1, but Rdevel is version 3.2. However, most, but not all of the packages in this directory are masked by /usr/local/lib/R/library.

Here are couple options:

• Set the libpath for Rdevel to just /usr/local/lib/R/library
• Set the libpath for Rdevel to /usr/local/lib/R/site-library-devel, /usr/local/lib/R/library. You'd need to create the site-library-devel directory.

[eddelbuettel]

It is on purpose as we do not want to install everything twice.

The other three entries are the standard Debian layout we had since 2003 or so and which is documented elsewhere.

[eddelbuettel]

@wch here is a brief 'what is where and why'

• /usr/local/lib/R/site-library/: Packages installed by the user, and we share this on purpose between R and Rdevel.
• /usr/lib/R/site-library/: By convention, use for .deb package not part of Base R. We just don't have any installed in this image, as soon as you apt-get or dpkg one you see it here
• /usr/lib/R/library/: As you noted, base packages for R installed via .deb like r-cran-mass.
• /usr/local/lib/R/library/ The build-local directory for Rdevel.

I consider the Docker containers to be 'temporary' enough so that one focused on R-devel will see use primarily of R-devel -- so this version of R can easily re-use earlier/other packages from R-release. This strikes me as easier and more lightweight than forcing each package in twice. Which you can still do simply by changing the .libPath() -- but this strikes me as a worse default.

[wch]

@eddelbuettel thanks for the explanation. The rationale for sharing /usr/local/lib/R/site-library makes sense (especially with the idea that containers are disposable), though it does result in the occasional problem where a package installed in one version of R is broken in another version.

You could still allow packages installed on R-release to be used on R-devel, while also ensuring that packages installed on R-devel aren't used on R-release, by setting the libpath to: /usr/local/lib/R/site-library-devel, /usr/local/lib/R/site-library, /usr/local/lib/R/library, /usr/lib/R/library. This seems to me to be a better strategy, and the cost is negligible.

We've been bitten by R version changes in the past: when R 3.0.1 was released and CRAN started building binary packages on it, any package that contained reference class objects (including Shiny) would no longer run on R 3.0.0. This is because in R 3.0.1, they added and used a function named .setDummyField: https://github.com/SurajGupta/r-source/blame/master/src/library/methods/R/refClass.R#L748. If you google for that string, you'll see a lot of references to problems that people had.

But as you said, this isn't a huge deal if you treat the containers as disposable, so I don't feel super strongly about it.