Workflow organization: guidelines and good practices

In this last section of the main part of the workshop, we would like to synthesize all we have learned about workflow organization and present some of the official guidelines from Snakemake's documentation. The goal of this section is that you are able to organize your own workflow in the best and most standard way possible.

Below is a list of recommendations:

• A repository should contain a single workflow (the workflow itself can handle multiple tasks)
• Use wrappers whenever available. If not, provide Conda environments when possible
• If your workflow consists of more than a few rules, break it down in several snakefiles
• Use the extension .smk for snakefiles (except for the main Snakefile)
• The following organization is recommended for a complete workflow repository:

.
├── config.yaml               # Config file defining parameter values for the workflow
├── envs                      # Directory for Conda environments 
│   ├── first_task.yaml       # Conda environment file
│   └── second_task.yaml
├── LICENSE      
├── README.md
├── rules                     # Directory for snakefiles implementing rules
│   ├── first_task.smk        # File implementing a subset of rules
│   └── second_task.smk
├── scripts                   # Directory for external scripts
│   └── script.py             # External script (Python, R, Julia ...)    
└── Snakefile                 # Main Snakefile for the workflow

• Define all possible parameters in a config file
• Use explicit rule and variable names; first_step was actually not a good name for a rule !
• Don't be shy to use comments to explain your workflow; use docstring comments in rules

---

If you follow these guidelines, you will be able to easily package your workflow in an archive by running snakemake --archive <name>.tar.gz. The archive will contain all code and config files under version control (git), as well as all input files, all environment files, and even the software packages from these conda environments. Consequently, the workflow can be reproduced entirely by decompressing the archive and running snakemake within the main directory.