Skip to content

HPC and SLURM

Ali Altintas edited this page Aug 26, 2026 · 1 revision

HPC and SLURM

Rule-level threads/memory/runtime defaults live in workflow/default-config/resources.yaml. Override any of them by creating input/resources.yaml with just the keys you want to change (partial overrides are deep-merged). Unlisted rules fall back to conservative defaults (1 thread / 4 GB / 60 min).

To submit to SLURM, run snakemake with the bundled workflow profile (from the repo root, like any snakemake invocation):

snakemake --workflow-profile workflow/profiles/slurm --configfile config/test.yaml   # smoke test on SLURM
snakemake --workflow-profile workflow/profiles/slurm --cores 64                     # limit total cores
snakemake --workflow-profile workflow/profiles/slurm -n                             # dry-run

The profile (workflow/profiles/slurm/config.yaml) submits every job with sbatch and sizes each one from input/resources.yaml. slurm_account in the profile is pre-set to the ICMM_DM group account; replace it (and qos) with your cluster's values if you're not in that group. slurm_partition is left unset on purpose, so sbatch uses the cluster's default partition (if yours has none, add slurm_partition). Per-invocation overrides are also possible, e.g. --set-resources star_align:mem_mb=64000. Every tool runs from the shared conda env, so make sure it's visible from the compute nodes — if nodes can't read your home dir, install on shared storage (conda env create -p /shared/path/env, or ./workflow/scripts/install-env.sh -o /shared/software/tetranscripts-pipe-env) and put its bin directory on the nodes' PATH.

Clone this wiki locally