mirtrace tries to allocate half of the installed RAM

[zeehio]

mirtrace can be executed in two ways: with the mirtrace wrapper script or calling the jar directly (java -jar mirtrace.jar...).

mirtrace requires a large amount of heap allocated on the java virtual machine. Its README states that the wrapper script sets the heap to half of the RAM of the system. For custom heap values, they suggest to not use the wrapper script and call the jar directly. See https://github.com/friedlanderlab/mirtrace

Currently this nexflow pipeline uses the mirtrace wrapper script, so it tries to allocate half of the RAM installed in the system. I'm trying to run this pipeline on a shared server with a lot of RAM installed but with a smaller RAM limit available for me. If the system has 1.4TB of RAM installed, mirtrace tries to allocate 700GB, but I have a limit of 300GB.

I made a pull request to mirtrace to let the user define the heap limit on the mirtrace wrapper as well friedlanderlab/mirtrace#4. In parallel, while my contribution is revised, I would like to report this issue and ask if you could please call the mirtrace jar directly with a reasonable heap value (maybe derived from the nextflow memory limit?). My nextflow skills are still very limited and I would appreciate a lot if someone could fix this (I would definitely learn from the solution) or give me an overdetailed explanation so I could fix it myself and submit a PR here.

My first successful attempt to get this fixed is at zeehio@27ec0cf.

It requires (1) find the location of mirtrace.jar and (2) call java directly.

For (1) I used the same approach used in the mirtrace wrapper script:

mirtracejar=\$(dirname \$(which mirtrace))

For (2) I replaced the mirtrace qc call (in the mirtrace process, in main.nf) with:

java -Xms4096M -Xmx4096M -jar \$mirtracejar/mirtrace.jar --mirtrace-wrapper-name mirtrace qc 

A proper solution would set -Xms4096M -Xmx4096M dynamically, based on the current nextflow memory limits. That's beyond my current nextflow skills but it should be easy for someone with experience.

Thanks in advance

PS: might be related to #44 but I can't tell from that issue if that's the case

[KevinMenden]

Hi @zeehio ,

I'm going to fix some bugs in the pipeline now anyway, which are also related to mirtrace, so I might as well take a shot at this issue. Thanks for reporting!

Don't have much hope for your PR over at mirtrace, seeing how little maintenance this tool seems to get :/

Cheers,
Kevin

[zeehio]

Hi Kevin! Thanks for the (partial) fix. mirbase --version will still try to allocate half of the RAM though

smrnaseq/main.nf

Line 251 in 976bb3e

mirtrace --version > v_mirtrace.txt

[KevinMenden]

Ah, thanks! Good catch .... 🤦 will fix that today