SLURM-GEN

Small Python script that generates SLURM batch scripts to run job arrays easily for a large number of different command-line parameters.

Usage example:

./slurm-gen.py -s time=00:00:01 -e OMP_NUM_THREADS=1 -o run.sh -u echorun -- echo [1-10] [1-10]

This will for example generate the following sbatch script:

#!/bin/bash -l
#SBATCH --time=00:00:01
#SBATCH --array=0-99%10
#SBATCH --output=echorun_%a.out

# sbatch script generated by slurm-gen using arguments:
# echo [1-10] [1-10]

varray0=(1 2 3 4 5 6 7 8 9 10)
varray1=(1 2 3 4 5 6 7 8 9 10)

r=${SLURM_ARRAY_TASK_ID}
d=$(($r/10))
i0=$(($r - $d*10))
r=$d
v0=${varray0[${i0}]}

d=$(($r/10))
i1=$(($r - $d*10))
r=$d
v1=${varray1[${i1}]}

if [ ! -s "echorun_${SLURM_ARRAY_TASK_ID}.out" ] || [ -n "$(grep -l 'srun: error' "echorun_${SLURM_ARRAY_TASK_ID}.out")" ]
then
    OMP_NUM_THREADS=1 srun echo ${v0} ${v1}
fi

When submitted to slurm, this file will run echo 1 1, echo 1 2, ..., echo 9 10, echo 10 10, in total 100 jobs with the given 100 different combinations of command line arguments. It supports a number of different ranges as well as multiple references of the same range. Run ./slurm-gen.py -h for documentation of all supported arguments and full range syntax.