Running Spades coassembly for Metageonomic datasets in a multinode mode

[SidPalmal4029]

Hello ,

I am currently working with some heavy metagenomic environmental datasets where the raw files are on average in the ballpark of 40 Gb. Currently, we are doing individual assembly. But some of our reviewers have suggested we use co-assemble with Spades. The main problem is our current server 96 C and 750 Gb RAM, is not capable of running the co assembly as we are hitting OOM error. We also have access to one HPC, but the problem is that we have node reservation for nodes , and each has a cap of 500 gb . So when we are trying to submit to the HPC via qsub and reserving 2 nodes, the job is getting terminated without any output. One of our inhouse expert suggested that Spades might not be supporing open MPI standards to submit a job to multi node. PLease enlighten me if thats the case or help me in this regard if people are submitting jobs on multinodes, and whats the trick to do it..

[asl]

But some of our reviewers have suggested we use co-assemble with Spades.

Frankly speaking co-assembly is in the most cases not the best idea. However, indeed some reviewers might insist on this. Overall there are lots of questions how one could interpret the results of co-assembly, etc.

One of our inhouse expert suggested that Spades might not be supporing open MPI standards to submit a job to multi node.

Well, per manual, vanilla SPAdes is intended to be run on a single node. If you run it across multiple nodes with the same parameters, then likely these two jobs will just overwrite each ones results and intermediate files with arbitrary bad outcome. hpcSPAdes is able to utilize multiple nodes of cluster, however, there are lots of important things to consider. Ensure things outlined in https://ablab.github.io/spades/hpc.html are well-understood. Note, that splitting work across multiple nodes will not in general reduce / split memory consumption.

[SidPalmal4029]

Well that confirms my nightmare... Anyways, thanks for the insight.