slurm at nextflow

[human-npc]

Hi,

i'am new to the Nextflow world, and I apologize in advance if this is a question with an easily answerable solution.
I would like to execute my nextflow pipeline on a Slurm cluster. Our Slurm cluster is composed of a submission node, a master node, and computing nodes. Is it feasible to initiate my nextflow pipeline on my local computer and send the job/process to the submission node, where it would then execute the sbatch command? Or do I need a ssh connection to the submission node server, install nextflow, and run my pipeline directly on the server where the slurm workload manager is installed?

Thanks in advance

[bentsherman]

What's the difference between the submission node and master node?

[colindaven]

On slurm, you have a master/head node that controls a cluster workers. It's likely not a good idea to let users on to this, but keep them on a submit node.

[bentsherman]

Nextflow needs to run for the duration of the workflow and maintain a connection with sbatch. So you could launch Nextflow from the submit node as long as long-running processes are supported. Nextflow itself doesn't take up much resources (unless you want to run many workflows concurrently), it just submits lots of jobs and waits on them.

If you launch from your laptop, even if you have some way to submit jobs directly from your laptop, you would have to keep your laptop running and connected to your slurm cluster for the entire workflow run.

[human-npc]

Thanks for your responses! This answered my question!