Execution profiles

In their simplest form, execution profiles are presets of execution parameter values. A profile is implemented as a directory <profile_name> containing at least one file named config.yaml. In this file, you can specify a value for any execution parameter with the usual yaml syntax <parameter_name>: <value>. For boolean flags, e.g. --use-conda, use values true or false.

Profiles are stored in a global configuration directory; on Linux, this directory is usually ~/.config/snakemake. To know the directory for your system, check the --profile section of Snakemake's help (snakemake --help).

The following example implements a profile named conda_parallel:

Directory structure:

.config/  # In the user's home
 └─ snakemake/
     └─ conda_parallel/
         └─ config.yaml

config.yaml

use-conda: true  # Use conda to handle dependencies
jobs: 6  # Run a maximum of 6 jobs at a time

---

Execution profiles are particularly useful when running Snakemake on cluster environments. For this usage, two execution parameters are particularly useful:

• cluster: we saw this parameter in the cluster execution for submission commands, but it can also be used to specify a wrapper script around job submission.
• cluster-status: similar to cluster but to check a job's status; can also take a command or a wrapper script.

We will not expand on submission / job status wrapper scripts as they are very advanced concepts, but they offer a lot of freedom to customize execution on clusters. There is not a lot of information about submission scripts on the official documentation; you can find examples of submission profiles for most schedulers in this official repository.

We also implemented our own profile including a well-documented submission script for SLURM, which is available here.