Rule dependencies

The core principle of Snakemake's execution is to compute a Directed Acyclic Graph (DAG) that summarize dependencies between all inputs and outputs required to generate the final desired output. For each job, starting from the jobs generating the final output, Snakemake checks if required inputs exist. If they do not, the software looks for a rule that generates the input; this process is repeated until all dependencies are resolved.

Let us look at this example from the Defining rules section:

rule first_step:
    input:
        'data/first_step.tsv'
    output:
        'results/first_step.txt'
    shell:
        'cp {input} {output}'

According to the rule's definition, the input data/first_step.tsv is required to generate the output results/first_step.txt. In this case, data/first_step.tsv was an original input file for the pipeline, hence it already exists, and the dependency is resolved.

Now, let us define a second rule:

rule second_step:
    input:
        'results/first_step.txt'
    output:
        'results/second_step.txt'
    shell:
        'cat {input} | grep "snakemake" > {output}'

To generate the output results/second_step.txt, this rule requires the input results/first_step.txt. Before the workflow is executed, this file does not exist; therefore, Snakemake looks for a rule that generates results/first_step.txt, in this case the first defined rule first_step. The process is then repeated for first_step as described above. After all dependencies are resolved, Snakemake generates the DAG.

Because of this process, by default, an output can only be generated by a single rule; otherwise, Snakemake cannot decide which rule to use to generate this output, and the rules are considered ambiguous. In practice, there are ways to deal with ambiguous rules, which are not covered in this basic section (but see the relevant section in the official documentation).

It is possible to refer to the output of a rule directly in another rule with the syntax rules.<rule_name>.output. The following example implements this syntax for the two rule defined above:

rule first_step:
    input:
        'data/first_step.tsv'
    output:
        'results/first_step.txt'
    shell:
        'cp {input} {output}'

rule second_step:
    input:
        rules.first_step.output
    output:
        'results/second_step.txt'
    shell:
        'cat {input} | grep "snakemake" > {output}'

One advantage of using this syntax is that an change in output name will be automatically propagated to rules that depend on it, i.e. the name only has to be changed once.