Deleted log file in directory() output after encountering error

[Marc-Ruebsam]

Snakemake version
5.7.1

Description
Using directory() as output and storing the log inside this directory will delete the log upon encountering an error.

This is an example rule:

rule example:
    input:
        "some.input"
    output:
        directory("output/directory")
    log:
        "output/directory/some.log"
    shell:
        "somecomand --input {input} --output_dir {output} 2> {log}" 

which failed with:

Error in rule example:
    jobid: 2
    output: output/directory
    log: output/directory/some.log (check log file(s) for error message)
    shell: somecomand --input some.input --output_dir output/directory 2> output/directory/some.log
        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Removing output files of failed job example since they might be corrupted:
output/directory
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: /path/to/.snakemake/log/2019-10-21T192621.555072.snakemake.log

and therefor deleted my log file inside the output directory.

I know it might be intended to save the log files inside a separate local directory like

    log:
        "log/example_rule_some.log"

but I prefer to keep my logs with the created files, if possible.

Possible Solution
check for {log} in {output} if {output} is a directory() and delete everything but {log}

Thanks!
Keep up the good work =)

Edit:
Linux examplePC 3.16.0-7-amd64 #1 SMP Debian 3.16.59-1 (2018-10-03) x86_64

[johanneskoester]

Thanks for reporting. Here, deleting the directory is mandatory, because the directory flag shall guarantee that it is completely created by the shell command of that rule. But as you say you create files, maybe you can have those files as output files, instead of just the directory. This way, the log can be in there as well.

It is best practice to have a separate log directory though, see here.