Skip to content

Resource Usage and Reports

Ali Altintas edited this page Aug 26, 2026 · 1 revision

Resource Usage and Reports

Every rule records its runtime and peak memory (threads/CPU-seconds) to results/pipeline_info/benchmarks/<rule>/.... Three ways to use them:

  • Inside the MultiQC report. Every run's qc/multiqc_report.html ends with a "Resource usage" section (aggregated from the benchmark files by the benchmark_summary rule) — a per-rule table of job count, mean/max wall time, and for both CPU and RAM: the allocated amount (resources.yaml), the mean/max actually used, and the efficiency (mean used / allocated). The quickest way to see which rules are the big hitters, how well you've sized their threads/memory, and where you can reclaim resources.

  • Per-rule cost tracking. Each benchmark file is Snakemake's own short table (written by Snakemake >=8): s, h:m:s, max_rss [MB], max_vms, max_uss, max_pss, io_in, io_out, mean_load [%], cpu_time. A quick way to list them:

    ls results/pipeline_info/benchmarks/*/*
  • Full execution report. Snakemake's own --report flag bundles the benchmark data plus the DAG, rules, and (with --report --sdm conda) environment details into a single HTML file — handy for sharing how a run consumed resources:

    ./workflow/scripts/benchmark-report.sh --configfile config/test.yaml   # -> report.html
    ./workflow/scripts/benchmark-report.sh -o out/report.html              # custom path
    snakemake --workflow-profile workflow/profiles/slurm --report report.html   # on SLURM

    workflow/scripts/benchmark-report.sh is a thin wrapper around snakemake --report <out> "$@" that passes everything else straight through. Because every rule declares benchmark:, the "Report" page lists peak CPU/memory per job; the --report HTML has a "Benchmarks" section with per-rule CPU/RSS charts.

Note: --report re-evaluates the whole DAG including outputs, so the benchmark files must already exist from a finished run (that's what it aggregates). Run the workflow once, then --report.

Clone this wiki locally