rule philosopher_database permissions

[cmkobel]

Config file config.yaml is extended by additional config specified via the command line.
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Select jobs to execute...

[Tue Jul 19 10:21:30 2022]
rule philosopher_database:
    output: output/220506_DnnT6S_carl/msfragger/philosopher_database.fas
    jobid: 0
    benchmark: output/220506_DnnT6S_carl/benchmarks/philosopher_database.tsv
    reason: Forced execution
    wildcards: config_batch=220506_DnnT6S_carl
    resources: mem_mb=1000, disk_mb=1000, tmpdir=/scratch/3973804




        >&2 echo "Catting database files ..."
        # Cat all database source files into one.
        cat  > output/220506_DnnT6S_carl/msfragger/cat_database_sources.faa


        >&2 echo "Change dir ..."
        # As philosopher can't specify output files, we need to change dir.
        mkdir -p output/220506_DnnT6S_carl/msfragger
        cd output/220506_DnnT6S_carl/msfragger

        >&2 echo "Philosopher workspace clean ..."
        /faststorage/project/gBGC/bin/philosopher_v4.1.1_linux_amd64/philosopher workspace             --nocheck             --clean 

        >&2 echo "Philosopher workspace init ..."
        /faststorage/project/gBGC/bin/philosopher_v4.1.1_linux_amd64/philosopher workspace             --nocheck             --init 

        >&2 echo "Removing previous .fas ..."
        rm *.fas || echo "nothing to delete" # Remove all previous databases if any.

        >&2 echo "Philosopher database ..."
        /faststorage/project/gBGC/bin/philosopher_v4.1.1_linux_amd64/philosopher database             --custom cat_database_sources.faa             --contam 

        >&2 echo "Move output ..."
        # Manually rename the philosopher output so we can grab it later
        mv *-decoys-contam-cat_database_sources.faa.fas philosopher_database.fas

        >&2 echo "Clean up ..."
        # clean up 
        rm cat_database_sources.faa


        
Catting database files ...
Change dir ...
Philosopher workspace clean ...
time="10:21:32" level=info msg="Executing Workspace  v4.1.1"
time="10:21:32" level=info msg="Removing workspace"
time="10:21:32" level=warning msg="Cannot read file. open .meta/meta.bin: no such file or directory"
time="10:21:32" level=info msg=Done
Philosopher workspace init ...
time="10:21:32" level=info msg="Executing Workspace  v4.1.1"
time="10:21:32" level=info msg="Creating workspace"
time="10:21:32" level=warning msg="Cannot verify local directory path. getwd: no such file or directory"
time="10:21:32" level=fatal msg="Cannot create meta directory; check folder permissions. stat .meta: no such file or directory"
[Tue Jul 19 10:21:32 2022]
Error in rule philosopher_database:
    jobid: 0
    output: output/220506_DnnT6S_carl/msfragger/philosopher_database.fas
    shell:
        


        >&2 echo "Catting database files ..."
        # Cat all database source files into one.
        cat  > output/220506_DnnT6S_carl/msfragger/cat_database_sources.faa


        >&2 echo "Change dir ..."
        # As philosopher can't specify output files, we need to change dir.
        mkdir -p output/220506_DnnT6S_carl/msfragger
        cd output/220506_DnnT6S_carl/msfragger

        >&2 echo "Philosopher workspace clean ..."
        /faststorage/project/gBGC/bin/philosopher_v4.1.1_linux_amd64/philosopher workspace             --nocheck             --clean 

        >&2 echo "Philosopher workspace init ..."
        /faststorage/project/gBGC/bin/philosopher_v4.1.1_linux_amd64/philosopher workspace             --nocheck             --init 

        >&2 echo "Removing previous .fas ..."
        rm *.fas || echo "nothing to delete" # Remove all previous databases if any.

        >&2 echo "Philosopher database ..."
        /faststorage/project/gBGC/bin/philosopher_v4.1.1_linux_amd64/philosopher database             --custom cat_database_sources.faa             --contam 

        >&2 echo "Move output ..."
        # Manually rename the philosopher output so we can grab it later
        mv *-decoys-contam-cat_database_sources.faa.fas philosopher_database.fas

        >&2 echo "Clean up ..."
        # clean up 
        rm cat_database_sources.faa


        
        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message

[cmkobel]

Attempted mitigation is to not create the directory with mkdir -p <dirname> but to let rule link_input make this dirname first, and then let rule philosopher database fill this dir.
In theory it shouldn't make a difference but I want to get along..

[cmkobel]

The problem might arise if link_input fails and deletes the dir while rule philosopher_database is resetting (clean/init). Though, I'm pretty sure that wasn't the case for this occasion of failing.

[cmkobel]

Haven't seen this error for long. Closing.