Skip to content

Use $TMPDIR to set temporaryr directories#195

Merged
Aratz merged 5 commits into
nf-core:devfrom
Aratz:feature/pna-2349/use_TMPDIR
Mar 17, 2026
Merged

Use $TMPDIR to set temporaryr directories#195
Aratz merged 5 commits into
nf-core:devfrom
Aratz:feature/pna-2349/use_TMPDIR

Conversation

@Aratz
Copy link
Copy Markdown
Contributor

@Aratz Aratz commented Mar 16, 2026

HPC clusters typically specify the scratch area to be used with the environment variable TMPDIR. This PR makes sure we use it, and fallback on /tmp if it is not defined.

PR checklist

  • This comment contains a description of changes (with reason).
  • Make sure your code lints (nf-core pipelines lint).
  • Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
  • Usage Documentation in docs/usage.md is updated.
  • CHANGELOG.md is updated.

@Aratz Aratz self-assigned this Mar 16, 2026
@Aratz Aratz marked this pull request as ready for review March 16, 2026 14:50
@Aratz Aratz requested a review from a team as a code owner March 16, 2026 14:50
Comment thread conf/container_env.config
env {
MPLCONFIGDIR = '/tmp/.config/matplotlib'
NUMBA_CACHE_DIR = "/tmp/.numba_cache"
MPLCONFIGDIR = '${TMPDIR:-/tmp}/.config/matplotlib'
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realized something about this. Is this evaluated at the processing node, or head node? I.e. if the tmpdir is set differently on each node we need to be sure this is set there, not at the head node.

Copy link
Copy Markdown
Contributor Author

@Aratz Aratz Mar 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question! I couldn't find anything convincing enough in the docs, but I run the pipeline with the test profile and checked the .command.run file:

nxf_container_env() {
cat << EOF
export MPLCONFIGDIR="\${TMPDIR:-/tmp}/.config/matplotlib"
export NUMBA_CACHE_DIR="\${TMPDIR:-/tmp}/.numba_cache"
export XDG_CACHE_HOME="\${TMPDIR:-/tmp}/.cache"
export XDG_DATA_HOME="\${TMPDIR:-/tmp}/.data"
export PIXELATOR_DUCKDB_TEMP_DIR="\${TMPDIR:-/tmp}"
export PYTHONNOUSERSITE="1"
export R_PROFILE_USER="/.Rprofile"
export R_ENVIRON_USER="/.Renviron"
export JULIA_DEPOT_PATH="/usr/local/share/julia"
EOF
}

This is fed to the docker container when it is run, so this is run on the processing nodes.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. Than it should work as we want it to.

Comment thread docs/usage.md Outdated
Co-authored-by: Johan Dahlberg <johan@uppsala-bioinformatics.se>
@Aratz Aratz merged commit 7baa67f into nf-core:dev Mar 17, 2026
22 checks passed
This was referenced May 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants