Running BIDS apps on Niagara

Parallel Processing Bids Apps in Niagara Compute Canada

Under construction by Eduardo Garza.

If you know about BIDS-APPS you know it uses Docker containers. This is fine but they won't run in clusters. If you want to run your jobs in parallel, you need to use Singularity instead, and for that you need to first download and convert the Docker container to a Singularity image.

First I made the images in my PC using the docker2singularity container. Instructions are here. If we can have the images in the /projects folder, then it gets easier.

Load modules

#Singulariry is now available without a module
module load qbatch

Create the joblist file

This will run your bids-app in parallel. For this example I used fmriprep from Poldrack's lab.

singularity run -B /scratch/m/mchakrav/egarza/projects/addimex_conn/bids:/bids_data -B /scratch/m/mchakrav/egarza/projects/addimex_conn/output_fmriprep:/output -B /home/m/mchakrav/egarza/bin/singularity_images:/license /home/m/mchakrav/egarza/bin/singularity_images/poldracklab_fmriprep-2018-09-10-bdf644f5bdfe.img --fs-license-file /license /bids_data /output participant --participant_label 130 > joblist

Run it using qbatch

qbatch -N bids_test --chunksize 1 --walltime=8:00:00 joblist