Revisiting RSPM and CRAN default repos

[slager]

Background

Many Rocker images use this repos setting, which is great for conveniently installing binary packages from RSPM:

> getOption('repos')
                                                         CRAN 
"https://packagemanager.posit.co/cran/__linux__/jammy/latest"

However, this will throw an error in devtools::check() similar to those reported here:

#532

#547

Current workaround

The work-around, as already documented here, is to temporarily switch the repos entry for CRAN:

#532 (comment)

https://rocker-project.org/images/versioned/r-ver#switch-the-default-cran-mirror

Possible improvement

This temporary work-around is not be necessary if getOption('repos') is instead set like this:

> options("repos")
$repos
                                                         RSPM 
"https://packagemanager.posit.co/cran/__linux__/jammy/latest" 
                                                         CRAN 
                                "https://cloud.r-project.org" 

The explanation from a posit employee is here:

https://community.rstudio.com/t/404-error-during-devtools-check/166426/8

Apparently, install.packages() tries the first repo, but devtools::check() looks specifically for the repo named "CRAN".

The rig project apparently uses this dual-repo method:

https://github.com/r-lib/rig/pull/53/files

This tests well for me using install.packages() (installs RSPM binaries as desired) and devtools::check() (looks up the required file on CRAN as desired).

It seems possible from the Posit forum post above that RSPM might add the missing files in the future, which could pre-empt the error, but it seems unknown when or if this will happen.

Raising the issue here in case the Rocker project might want to consider implementing this, which at least on the surface seems very convenient. There may be potential unintended consequences of doing it this way, which I have not tested or thought through very thoroughly, but that the rig project implements it suggests that it could at least be worth considering.

[eitsupi]

Hi, thanks for thinking about this project.

I have thought about that of mine too.
But the purpose of rocker/r-ver is to provide a reproducible R environment easily¹, and I noticed that setting something like options(repos = c(CRAN = "https://cloud.r-project.org", RSPM = "https://packagemanager.rstudio.com/cran/__linux__/jammy/2023-01-29")), which is always up-to-date, is not match the purpose.
So I don't think we will make such a thing the default setting.

Of course, users who use rocker/r-ver for developing R packages, including myself, can change the settings at any time, and it should be well documented how to do so.
PR to the documentation would be welcome!

Footnotes

1. https://journal.r-project.org/archive/2017/RJ-2017-065/index.html ↩

[cboettig]

@slager Are you sure this is correctly diagnosed? I use devtools::check() from rocker images on a daily basis and can't reproduce the issue.

[eddelbuettel]

I had the same initial read as @eitsupi: r-ver is for point in time consistency. Yet it looks like you want 'perfect ability to test right now' with current CRAN.

And for what it is worth I also check current CRAN (which seems to be your goal) from the non-versioned stack e.g. in the r2u all the time for CI, debugging, and other testing uses. r2u starts from RSPM/PPM and makes full-fledged .deb binary package which get handled by apt with full dependencies resolution relying on bspm to dispatch from install.packages(). Last time I counted we had shipped over four million packages that way in a year and now routinely dispatch well over 10k per (business) day. RSPM/PPM are very powerful in providing a consistent installation base across different OS, archictectures and releases.

But in the case of Ubuntu LTS releases on x86_64, one can improve over it (as do some related projects for other Linux distros) and r2u does just that for the fairly common use case of running on Ubuntu LTS.

[slager]

I'm using a latest version from the versioned stack, so its RSPM repo is set to latest, not a specific date.

See here:
https://github.com/rocker-org/rocker-versioned2/wiki/Versions#rules
"If that R version is the latest, the CRAN date will not be set and the latest packages will always be installed"

My suggestion for adding CRAN = "https://cloud.r-project.org" as the second item in options('repos') is thus not to get more recent packages, but simply to suppress the devtools::check() error.

[eitsupi]

My suggestion for adding CRAN = "https://cloud.r-project.org" as the second item in options('repos') is thus not to get more recent packages, but simply to suppress the devtools::check() error.

As a maintainer, I am reluctant to complicate things just to suppress that error.
I suppose a developer could always use something like Makefile to run options(repos = c(CRAN = "https://cloud.r-project.org")) before running devtools::check().

[slager]

This is similar to what is already in the documentation, so if it is the preferred solution, feel free the close the issue. Thank you for the conversation.

[eddelbuettel]

Also if adding a command before running check is too much bother you can always use of the containers that do not need the step. I just picked a random (recent) github repo from another developer I had checked out and ran

docker run --rm -ti -v $PWD:/opt -w /opt rocker/r2u:jammy \
             Rscript -e 'install.packages("devtools"); devtools::install_deps("."); devtools::check()'

which installs 90 or so packages for devtools, then the two dependencies of the package I was testing and proceeded to test the package in question (and then failed because I forgot to tell it to skip vignettes -- no texlive in that container).

So in short this works without extra repo settings albeit not with the container you picked.

[slager]

Thanks. I benefit from using rocker/ml-verse:latest because of the pre-installed CUDA functionality, which the python part of my R/Python pipeline requires. At some point I'll look into setting up r2u starting with a CUDA base image.

[eitsupi]

Definitely the install.packages() and devtools::check() information you have researched is useful and I have opened a PR to update the document. (rocker-org/website#96)

I am closing this issue, but would appreciate your continued contribution by sharing information. Thanks!

[eddelbuettel]

Good point, and acknowledged! I should (time permitting) see about

• adding RStudio Server atop r2u
• adding CUDA atop r2u

but especially the second one is non-trivial. A decent (even if short) tutorial write-up might be a start.

[eitsupi]

Good point, and acknowledged! I should (time permitting) see about

* adding RStudio Server atop r2u

* adding CUDA atop r2u

but especially the second one is non-trivial. A decent (even if short) tutorial write-up might be a start.

(This is off-topic, but) the script https://github.com/rocker-org/devcontainer-features/blob/c926012d403621a83e9c3ab448188819e098033d/src/r-apt/install.sh should actually work on the CUDA image to set up r2u.
However, this is intended to be used as a Dev Container Feature, so it is not a good idea to run it directly.

[eddelbuettel]

Ah, yes, and as an Emacs user I should remember that I was also asked by a VS Code user to show devcontainers (which I don't use). So that really is a third bullet point.

[slager]

One additional thing that might be useful would be to feature the r2u images more prominently on https://rocker-project.org

[eitsupi]

One additional thing that might be useful would be to feature the r2u images more prominently on rocker-project.org

Definitely.....
I opened an issue rocker-org/website#97

[eddelbuettel]

Yeah. Been meaning to do that too but I also do not want to jump on @eitsupi's toes as r-ver seems to have its audience.

We have a bit of a "branding" issue as the stacks are somewhat non-overlapping so it gets confusing real fast.