Memory errors for SNP_EFF step

[ksw9]

Hi all,

Thank you for your work on this pipeline! @hpatel, thank you for your help on Slack, you suggested creating an issue if I couldn't resolve this with earlier suggestions.

I am running a the Java memory error with SNPEff and was wondering if you have suggestions for how to fix this. I updated my config file to expand the memory.

My command is:

nextflow run nf-core/viralrecon -r 2.0 -resume serene_gates --input /labs/jandr/walter/lambda/nf-core/sample_list/sample_list_050421.csv -profile singularity --platform illumina --protocol amplicon --outdir resultsV2 --email kwalter@stanford.edu --fasta /labs/jandr/walter/lambda/refs/MN908947.3.fasta --gff /labs/jandr/walter/lambda/refs/MN908947.3.gff --save_reference True --primer_bed /labs/jandr/walter/lambda/refs/artic_v3_primers.bed.gz --primer_fasta /labs/jandr/walter/lambda/refs//artic-ncov2019_v3_primers.fasta.gz --primer_set artic --primer_set_version 3 --kraken2_variants_host_filter True --kraken2_assembly_host_filter True --filter_duplicates False --skip_markduplicates True --skip_assembly True --max_cpus 8 --max_memory 200.GB --max_time 36.h --skip_asciigenome --skip_variants_quast -c /labs/jandr/walter/lambda/scripts/nf_custom.nf

My config file:

params {
    modules {
          'illumina_ivar_variants' {
            args = '-t 0 -q 20 -m 10'
          }
    }
}
process {
    withName: SNPEFF_ANN {
        memory = 200.GB
    }
}

The error message:

-[nf-core/viralrecon] Pipeline completed with errors-
Error executing process > 'NFCORE_VIRALRECON:ILLUMINA:VARIANTS_IVAR:SNPEFF_SNPSIFT:SNPEFF_ANN (0-10e3_S17)'

Caused by:
  Process `NFCORE_VIRALRECON:ILLUMINA:VARIANTS_IVAR:SNPEFF_SNPSIFT:SNPEFF_ANN (0-10e3_S17)` terminated with an error exit status (255)

Command executed:

  snpEff MN908947.3 \
      -config snpeff.config \
      -dataDir snpeff_db \
       \
      0-10e3_S17.vcf.gz \
      -csvStats 0-10e3_S17.snpeff.csv \
      > 0-10e3_S17.snpeff.vcf
  mv snpEff_summary.html 0-10e3_S17.snpeff.summary.html
  
  echo $(snpEff -version 2>&1) | sed 's/^.*SnpEff //; s/ .*$//' > snpeff.version.txt

Command exit status:
  255

Command output:
  (empty)

Command error:
  INFO:    Converting SIF file to temporary sandbox...
  WARNING: Skipping mount /var/singularity/mnt/session/etc/resolv.conf [files]: /etc/resolv.conf doesn't exist in container
  Error: Error while processing VCF entry (line 22720) :
  	MN908947.3	25320	.	GNNNNNNNN	G	.	FAIL	DP=5182	GT:REF_DP:REF_RV:REF_QUAL:ALT_DP:ALT_RV:ALT_QUAL:ALT_FREQ	1:5175:1958:37:1:0:20:0.000192976
  java.lang.OutOfMemoryError: Java heap space
  java.lang.OutOfMemoryError: Java heap space

I have included the nextflow.log. Please let me know if you have any suggestions or need any other information. Thank you again for your help!
Katie

[drpatelh]

Hi @ksw9 ! I think I have fixed this in #205

Be awesome if you can test this once it is merged.

nextflow pull nf-core/viralrecon
nextflow run nf-core/viralrecon -r dev <OTHER_PARAMETERS>

You shouldn't have to use a custom config anymore either to increase the memory requirements because they should be high enough by default. I suspect it was failing for you because you were calling "all" variants when using iVar (-t 0) and so snpEff ran out of memory.

[drpatelh]

Merged. Please feel free to re-open if the issue persists.

[ksw9]

Fantastic, thank you!!