Skip to content

Renv and dependencies

Jump to bottom Edit New page
Vini Salazar edited this page Aug 3, 2026 · 6 revisions

Workbench lessons keep two completely separate sets of R packages, and almost every confusing error people hit comes from accidentally merging them. This page explains the split, gives the one-time machine setup, and covers running Python inside an R Markdown lesson.

The one rule

The build toolchain is global. The lesson dependencies are managed by sandpaper. You never activate renv yourself.

Lives in Contains Installed by
Build toolchain your normal user library (.libPaths()[1], e.g. ~/Library/R/arm64/4.5/library) sandpaper, pegboard, tinkr, varnish you, once, with pak
Lesson dependencies renv/profiles/lesson-requirements/renv/library/ inside the lesson repo only the packages the .Rmd episodes actually library() sandpaper::manage_deps(), automatically

sandpaper deliberately does not make itself the project's renv environment. It uses a named renv profile called lesson-requirements, and it activates that profile in subprocesses, not in your session. serve() and build_lesson() spawn a callr subprocess for manage_deps() and one more per episode for knitting, each with RENV_PROFILE=lesson-requirements set, so .libPaths() inside those subprocesses is the lesson library. Your console keeps your normal library throughout, which is why sandpaper::serve() can find sandpaper at all.

So renv is in use during a build even though nothing in your session looks activated. If a build complains about missing packages, that is the episode subprocess reporting on the lesson library, not evidence that renv was skipped. See the build says packages are missing below.

If you run renv::init() or renv::activate() in a lesson repo you break this. Both commands write a .Rprofile at the repo root containing source("renv/activate.R"), which activates the lesson-requirements profile in every R session started in that directory. .libPaths() collapses to the lesson library, and sandpaper, pegboard and varnish appear to vanish. Do not run them.

What is in renv/, and what gets committed

renv/
├── activate.R                                  COMMIT   renv bootstrap, sandpaper calls it
├── profile                                     COMMIT   one line: "lesson-requirements"
├── .gitignore                                  COMMIT   keeps local scratch out of git
└── profiles/
    └── lesson-requirements/
        ├── renv.lock                           COMMIT   the pinned dependency set
        └── renv/
            ├── settings.json                   COMMIT
            ├── settings.dcf                    COMMIT
            ├── .gitignore                      COMMIT
            └── library/                        ignored, rebuilt on demand

renv.lock is the one that matters. CI decides whether a lesson has pinned dependencies purely by whether that file exists in the repository. If it is not committed, the lesson builds against whatever is current on the day, which is how a lesson silently stops reproducing.

A Markdown-only lesson (no .Rmd episodes, nothing executes) has no lesson dependencies and needs none of this. In those repos renv/ is gitignored on purpose.

One-time setup on your machine

We use pak rather than devtools. devtools has been split up and superseded, and pak resolves system requirements and GitHub sources without a separate install_github() call.

install.packages("pak")

options(repos = c(
  carpentries = "https://carpentries.r-universe.dev",
  CRAN        = "https://cloud.r-project.org"
))

pak::pak(c("sandpaper", "pegboard", "tinkr"))
pak::pak("melbournebioinformatics/uom-varnish")   # installs under the package name `varnish`

uom-varnish is our fork of the Carpentries varnish theme. It keeps the upstream package name, so it replaces varnish in your library rather than sitting alongside it. Every MB lesson pins it in config.yaml:

varnish: 'melbournebioinformatics/uom-varnish@main'
url: 'melbournebioinformatics.github.io/uom-varnish'

Then grant renv cache consent once, which sandpaper asks for interactively the first time otherwise:

sandpaper::use_package_cache(prompt = FALSE)

You also need pandoc and git on your PATH. Conda or miniforge is the easiest route on macOS.

Day-to-day

sandpaper::serve()          # live-reload preview on http://127.0.0.1:4321
sandpaper::build_lesson()   # one-off build into site/
sandpaper::check_lesson()   # pegboard structure and link validation

serve() and build_lesson() both call sandpaper::manage_deps() first, which:

  1. creates the lesson-requirements profile if it is missing,
  2. hydrates it from your user library, so packages you already have are linked rather than re-downloaded,
  3. downloads whatever is still missing,
  4. restores anything pinned in renv.lock,
  5. snapshots the result back to renv.lock.

Adding a package to an episode: install it into your user library the way you normally would, write library(thepackage) in the .Rmd, rebuild, then commit the changed renv.lock. Do not touch renv directly.

The install-it-first part matters. manage_deps() snapshots what is already installed; it will not go and fetch a package it has never seen. If the package is missing it says

The following required packages are not installed:
- clusterProfiler
Packages must first be installed before renv can snapshot them.

and quietly leaves it out of the lockfile, at which point CI has nothing to install and the episode fails to knit. Install first:

pak::pak("thepackage")                       # CRAN or GitHub
BiocManager::install("clusterProfiler")      # Bioconductor

For a Bioconductor lesson, make sure the lockfile carries a "Bioconductor": {"Version": "..."} key so renv resolves against the right release. It is written on the first snapshot taken with BiocManager available.

Useful maintenance calls:

sandpaper::manage_deps()                  # force the dependency pass on its own
sandpaper::update_cache()                 # update lesson packages, prompts per package
sandpaper::pin_version("ggplot2@3.5.1")   # hold a package at a version
sandpaper::no_package_cache()             # opt out of renv for this lesson entirely
options(sandpaper.use_renv = FALSE)       # opt out for this session only

renv::restore(), renv::init(), renv::activate() and renv::snapshot() are not part of this workflow. The only renv function worth knowing is renv::repair(), below.

Troubleshooting

could not find function "serve", or sandpaper/pegboard/varnish suddenly missing. There is a .Rprofile at the repo root. Delete it and restart R:

renv::deactivate(clean = TRUE)   # or simply: file.remove(".Rprofile")

Then check .libPaths() points at your user library again.

renv 1.1.5 was loaded from project library, but this project is configured to use renv 1.0.11. Same cause: the project got activated. Same fix.

The following package(s) have broken symlinks into the cache. The global renv cache moved or was cleaned. Either renv::repair(), or delete renv/profiles/lesson-requirements/renv/library and let the next manage_deps() rebuild it.

The project is out-of-sync -- use renv::status() for details. Only meaningful if you activated the project. Deactivate, then rebuild the lesson normally.

The build says packages are missing

Three different failures look alike. Match the message:

Packages must first be installed before renv can snapshot them, followed by a list. manage_deps() only snapshots packages that already exist; it will not fetch one it has never seen. They are silently left out of the lockfile, so the profile library never gets them and the episode fails to knit. Install them into your user library first, then rebuild:

pak::pak("thepackage")
BiocManager::install("clusterProfiler")

there is no package called 'X' during knitting. Same cause, one step later: the episode subprocess is running against the lesson library and X is not in it. Fix it the same way.

aborting snapshot due to pre-flight validation failure, listing packages as "required but not installed". Typically cachem, evaluate, fastmap, fontawesome, jquerylib, memoise, mime and sass. Nothing is wrong with the lesson. Those packages exist only in your R system library, so renv::restore() sees them on .libPaths() and skips installing them, while renv::snapshot() validates against the project library alone and reports them missing. The build gets stuck between the two.

Tell renv the system library is a legitimate external source. This is exactly what the CI container does, via the same variable:

Sys.setenv(RENV_CONFIG_EXTERNAL_LIBRARIES = .Library)
sandpaper::build_lesson()

On macOS .Library is /Library/Frameworks/R.framework/Versions/<ver>-arm64/Resources/library. To make it permanent, put that Sys.setenv() line in your user-level ~/.Rprofile. That file is fine and normal. The one you must never create is a .Rprofile inside a lesson repository.

The lockfile records a different R version than the one you are running. A warning, not an error. renv proceeds and the version is re-recorded on the next snapshot. It happens whenever a lesson was last built by someone on a different R release.

CI builds fine but nothing is pinned. renv.lock is not committed. Check git ls-files renv lists it, and check .gitignore does not contain a bare renv/ line.

Bioconductor-heavy lessons take forever on first build. Expected. manage_deps() is compiling into a fresh profile library. Subsequent builds link from the renv cache and are fast.

Python in an R Markdown lesson

R Markdown episodes can run Python chunks through reticulate. Since reticulate 1.41 you no longer manage a conda or virtualenv by hand: py_require() declares what the episode needs and reticulate provisions an ephemeral environment with uv, downloading both the interpreter and the packages.

Put this in the first chunk of the episode:

```{r setup, include=FALSE}
Sys.setenv(RETICULATE_PYTHON = "managed")   # ignore any conda/system python on PATH
library(reticulate)
py_require(
  c("pandas", "matplotlib"),
  python_version = "3.12",
  exclude_newer  = "2026-08-01"             # freeze resolution to this date
)
```

Then write Python chunks normally, and cross the language boundary with py$name from R and r.name from Python:

```{python}
import pandas as pd
df = pd.DataFrame({"x": [1, 2, 3], "y": [4, 5, 6]})
df.describe()
```

```{r}
summary(py$df)
```

Three things worth knowing:

  • RETICULATE_PYTHON = "managed" is not optional in practice. reticulate walks an order of discovery and uses the ephemeral environment only if it finds nothing earlier. On a machine with miniforge on the PATH, conda wins and your py_require() declarations are silently ignored.
  • renv.lock records R packages only and will never record Python packages. exclude_newer is the lightweight substitute: it caps package resolution at a date, so a rebuild next year still resolves the versions that were current when you wrote the episode. Pin exact versions ("pandas==2.2.3") if you need more than that.
  • This needs no change to CI. reticulate fetches uv into its own cache inside the carpentries/workbench-docker container, and uv fetches the interpreter. There is nothing to add to the workflow files.

reticulate is an R package, so manage_deps() picks it up into renv.lock automatically once an episode calls library(reticulate).

When to use conda instead

py_require() resolves from PyPI. If a lesson needs a heavyweight bioconda stack, or tools that are not on PyPI at all, fall back to a conda environment:

  1. Commit an environment.yml at the lesson root.

  2. In the setup chunk, point at it explicitly rather than using "managed":

    Sys.setenv(RETICULATE_PYTHON = "/opt/conda/envs/lesson/bin/python")
library(reticulate)

  3. Add a conda setup step to .github/workflows/docker_build_deploy.yaml that creates the environment at that path before the build step runs.

This is more work and more to maintain, so use it only when PyPI genuinely cannot supply the tools. Also consider whether the episode should be showing terminal output rather than executing a bioinformatics pipeline at build time.

How CI uses all this

Worth knowing when a build behaves unexpectedly:

  • carpentries/actions/renv-checks sets renv-needed from the presence of renv/profiles/lesson-requirements/renv.lock.
  • The GitHub Actions cache key is a hash of that lockfile, scoped by OS and R version, since packages are compiled for both.
  • The build container runs with RENV_PROFILE=lesson-requirements, RENV_PATHS_ROOT=/home/rstudio/lesson/renv and RENV_CONFIG_EXTERNAL_LIBRARIES=/usr/local/lib/R/site-library, so it reuses the toolchain baked into the image and only installs the lesson's own packages.
  • The scheduled workflow 02 Maintain: Check for Updated Packages opens a PR titled "Update N packages" that bumps renv.lock. Merge these. A comment on the PR shows how the rendered output changed, so you can see whether an update broke anything before it reaches main.
  • Workflow files themselves are updated by sandpaper::update_github_workflows() or the 04 Maintain: Update Workflow Files action.

Further reading

Add a custom footer
Add a custom sidebar

Clone this wiki locally