How do you handle reproducibility and interoperability when working with R

[mahesh-panchal]

There seem to be several issues when it comes to working with R that hinder reproducibility and interoperability.

• Packages on Cran/Bioconductor aren't packaged in the same way Python or Conda packages are handled, and may need to be compiled for your OS.
• There's an effort to make binaries of R packages on conda-forge, but these could be months behind the latest version of a package you need.
• Using renv (or other package managers) aren't cross platform and build for your OS, which also means you may end up in dependency hell if something is missing / not found/ incorrectly linked.

The most common solution I see to this is that people propose Dockerising everything so you snapshot the packages in the image and are always working on whatever the image is based on. However building can take time, and not everyone has access to a system where they may be able to build docker images. Dependency issues can still be a problem then.

And what about working on HPC? Is this an effective way to work when you want to run R on a HPC node. Do you build locally and convert to singularity on the HPC? Is there a smooth way to do this for sensitive data environments?

Some people also want to use polyglot environments for example mixing both python and R. Some package managers handle this by offloading to other package managers. Should this be dockerised then too, and then how big is the resulting image (which can cause conversion problems, etc)?

What's been the smoothest way for you to work with R?

[CormacKinsella]

I'm not a regular R user, but one approach I have used for a pipeline running several R packages was to host a Docker image via GHCR on the code repo, and then use Apptainer on HPC systems to pull & run. In the pipeline I link to the latest tag.

[mahesh-panchal]

Resources for best practices:

• https://collabnix.com/10-essential-docker-best-practices-for-r-developers-in-2025/
• https://github.com/rsangole/user2022-r-for-docker

For setting up a devcontainer:

• https://github.com/rocker-org/devcontainer-try-r

Why R packaging is so difficult

• Suggestion: save version and hash of packages by default A2-ai/rv#232 (comment)

[mahesh-panchal]

Seqera's Wave also has the capacity to build images from Cran

https://github.com/seqeralabs/wave-cli

  # Build a container based on CRAN packages
  wave --cran-package dplyr=1.1.0 --cran-package ggplot2

  # Build a container based on CRAN packages with custom R base image
  wave --cran shiny --cran-base-image rocker/tidyverse:4.3.0

  # Build a container based on CRAN packages using arm64 architecture
  wave --cran dplyr --platform linux/arm64

[fasterius]

I find that as long as I'm not requiring the absolute latest packages, working with Conda/Pixi is a good solution. I still containerise everything, though, but Conda/Pixi remains a highly convenient installation method even inside images. I also tend to store them on GHCR of the code repo, like Cormac mentioned. I also always run inside Nextflow pipelines, so I don't have to care about the images regardless of what platform/HPC I'm on.

When requiring the very latest versions just as they've been released is another matter, though. While I feel that Bioconductor is generally good about automating this process, there's still a lag period of some time - maybe a month or so, anecdotally? I do feel that all this is more due to R as a language just being weird when it comes to installing packages and the logistics around that. Don't get me wrong, I love many things about R, but packaging is just horrible in general.

[juliahoglund]

oh wow, I just updated a really messy Dockerfile for an R package to run. I wish I saw the best practices earlier! First time I did it I did it with renv and using that one in a Docker, however, I see now that it might be build specific, and I myself only tested it on one HPC (i.e. one type of architecture).

the way I solved it now was super tedious, by both locking package versions to a CRAN snapshot, adding versions to all apt packages, adding some extra dependencies manually through apt and BiocManager and some other workaround, and now it seems to work at least as Dockerfile through docker, as singularity image and on amd64 and arm64. But I have no idea whether it is fully locked yet honestly.

only problem now, is as mentioned above that it took quite some time to build the image in the end.

are there thoughts on having R best practices (or similar) as a tech short presentation in the future?

[mahesh-panchal]

are there thoughts on having R best practices (or similar) as a tech short presentation in the future?

I don't think there are any issues / walkthrough suggestions for that yet. Please feel free to make one. I think there are several people interested in this and we could get on the schedule soon ( soon... February )

[mahesh-panchal]

I made an issue for it: #148

[machlabd]

Hi! Unless someone else wants to, I am happy to host/co-host a session on this. I can give an overview of what at least my "best" practices have been so far for working reproducibly with R, especially when working interactively, and we can have a discussion on if there are better ways. I haven't used containers for this, but I also wonder if that makes more sense once one is at the later/final stages of an analysis where one knows exactly which packages/package versions are needed.
Here are some suggestions of what could be covered:

• Bioconductor releases, and installing with BiocManager::install
  • e.g.: R-BioC/4.5.0-3.21
  • how are other R packages dealt with here (CRAN/GH)
  • when to stop updating
  • options on different platforms: Mac vs compute server (less specific on Mac, without the 3rd digit of the R version)
  • launching different R+packagePath versions on Mac (rig, .libPaths(), RStudio), and on compute server
  • R-bioc/dev
• other (I haven't used but happy to explore): renv
• overall best practices: running .Rmd/.qmd file with snakemake (using the setup described above), sessionInfo()/date()

Let me know if you think this could be interesting or fits what you had in mind.

[addityea]

Wonderful!
My two cents about containers/ Pixi (which I push for all the time, so probably biased):
I have generic version controlled Docker containers for mundane analysis with routine packages, but if a project is rather small and requires only a few packages, I end up using Pixi local at the beginning of the project and then a Docker/ Apptainer with the same Pixi env, ensuring reproducibility and solving the need for constantly updating the containers... Never really used Renv

Rest, sounds like a good walkthrough! :)

[mahesh-panchal]

That sounds good. I'll assign you to #148.

[machlabd]

Sounds good!

[machlabd]

Thanks Adi for the feedback! If you/others have anything else to add please feel free to mention them. I can't promise to cover everything but would be good to know what to potentially dive into.