feat: add Bash script functionality

[mbhall88]

Description

This PR adds the ability to access snakemake/rule variables in Bash scripts [closes #294].

There are some design choices here that deserve some discussion. And I will update the docs when/if it is agreed that this PR is desirable.

The main thing to know is that the interface with the snakemake object in the Bash script is a little different to other languages we support.

We use associative arrays (AAs) to provide access to rule variables, but you can't nest AAs. As such, there is not a single snakemake variable (AA), but one for each rule attribute that is encoded as a Namedlist (i.e., input, output, wildcards, log, params etc.), and then a "master" variable for everything else (i.e., threads, rule name, scriptdir etc.).

The other thing to know is we cannot fully support all python data structures. For example, if you have a dict as the value for a variable in params, it cannot be correctly represented, as it would require us to nest an AA.

In addition, lists (of say files) are encoded as space-separated strings - because again, you can't nest arrays.

Despite these limitations, I think the functionality is still super useful, and I know it will save me a lot of effort in my workflows.

Example

Here is an example script that we create for the Bash test rule I added

#!/usr/bin/env bash

SNAKEMAKE_INPUT=( [0]="test2.in" [1]="test.in" [named_input]="test.in" )
SNAKEMAKE_OUTPUT=( [0]="bash.out" )
SNAKEMAKE_PARAMS=( [0]="123" [integer]="123" [1]="foo" [string]="foo" [2]="a" [d]="a" )
SNAKEMAKE_WILDCARDS=( )
SNAKEMAKE_RESOURCES=( [0]="1" [_cores]="1" [1]="1" [_nodes]="1" [2]="/var/folders/0g/3ywwrf6j2vjbp0r780dskt980000gn/T" [tmpdir]="/var/folders/0g/3ywwrf6j2vjbp0r780dskt980000gn/T" [3]="1024"
[mem_mb]="1024" )
SNAKEMAKE_LOG=( [0]="bash.log" )
SNAKEMAKE_CONFIG=( [test]="True" )
SNAKEMAKE=( [threads]="1" [rule]="bash" [bench_iteration]="None" [scriptdir]="/var/folders/0g/3ywwrf6j2vjbp0r780dskt980000gn/T/snakemake-9uo9vmd2/scripts" )

echo "The first input file is ${SNAKEMAKE_INPUT[0]}" > "${SNAKEMAKE_OUTPUT[0]}" 2> "${SNAKEMAKE_LOG[0]}"
echo "The named input file is ${SNAKEMAKE_INPUT[named]}" >> "${SNAKEMAKE_OUTPUT[0]}" 2>> "${SNAKEMAKE_LOG[0]}"
echo "The requested number of threads is ${SNAKEMAKE[threads]}" >> "${SNAKEMAKE_OUTPUT[0]}" 2>> "${SNAKEMAKE_LOG[0]}"

I'm happy to take feedback and make any changes, but this is a good starting point I think.

QC

• The PR contains a test case for the changes or the changes are already covered by an existing test case.
• The documentation (docs/) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).

[johanneskoester]

Nice! I like the approach! Feel free to finalize and let me know once it should be reviewed in detail.

One thing to note immediately: I think the variables should be lower case. The upper case seems a bit heavy on the eye.

echo "The first input file is ${SNAKEMAKE_INPUT[0]}" > "${SNAKEMAKE_OUTPUT[0]}" 2> "${SNAKEMAKE_LOG[0]}"
# vs
echo "The first input file is ${snakemake_input[0]}" > "${snakemake_output[0]}" 2> "${snakemake_log[0]}"

[mbhall88]

Okay @johanneskoester, I've made the variables lowercase and added documentation with two examples featuring some of the slightly different use compared to other languages' scripts.

One thing, the windows tests seems to be failing as Bash doesn't seem to be a conda package for windows? Not sure what to do about that...

[johanneskoester]

Very nice work!

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
1 Code Smell

No Coverage information
0.0% Duplication