Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pilon step runs out of error (java.lang.OutOfMemoryError) #147

Closed
RaverJay opened this issue Oct 15, 2018 · 2 comments
Closed

Pilon step runs out of error (java.lang.OutOfMemoryError) #147

RaverJay opened this issue Oct 15, 2018 · 2 comments

Comments

@RaverJay
Copy link

RaverJay commented Oct 15, 2018

Hey,

a unicycler run threw the following error on the pilon step:

Polishing assembly with Pilon (2018-10-12 15:50:32)
Unicycler now conducts multiple rounds of Pilon in an attempt to repair any remaining small-
scale errors with the assembly.

Aligning reads to find appropriate insert size range...
Insert size 1st percentile:  134
Insert size 99th percentile: 567

Pilon polish round 1
Total number of changes: 40

Pilon polish round 2
Total number of changes: 12

Pilon polish round 3
Total number of changes: 9

Pilon polish round 4
Total number of changes: 2

Pilon polish round 5
No Pilon changes

Pilon polish round 6
Unable to polish assembly using Pilon: Pilon encountered an error:
Pilon version 1.22 Wed Mar 15 16:38:30 2017 -0400
Genome: 6_polish_input.fasta
Fixing snps, indels, gaps, local
Input genome size: 975500
Scanning BAMs
6_paired_alignments.bam: 16290524 reads, 0 filtered, 15509205 mapped, 14678436 proper, 789568 stray, FR 100% 278+/-86, max 536
Processing 3:1-12069
frags 6_paired_alignments.bam: coverage 3330
Total Reads: 418549, Coverage: 3330, minDepth: 333
Confirmed 12067 of 12069 bases (99.98%)
Corrected 0 snps; 0 ambiguous bases; corrected 0 small insertions totaling 0 bases, 0 small deletions totaling 0 bases
# Attempting to fix local continuity breaks
# fix break: 3:11297-11602 0 -0 +0 NoSolution TandemRepeat 48
3:1-12069 log:
Finished processing 3:1-12069
Processing 1:1-718813
frags 6_paired_alignments.bam: coverage 1511
Total Reads: 10898551, Coverage: 1511, minDepth: 151
Confirmed 718573 of 718813 bases (99.97%)
Corrected 0 snps; 0 ambiguous bases; corrected 0 small insertions totaling 0 bases, 0 small deletions totaling 0 bases
# Attempting to fix local continuity breaks
# fix break: 1:1586-1932 0 -0 +0 NoSolution TandemRepeat 30
Exception in thread "main" java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.simontuffs.onejar.Boot.run(Boot.java:340)
	at com.simontuffs.onejar.Boot.main(Boot.java:166)
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
	at java.util.Arrays.copyOfRange(Arrays.java:3664)
	at java.lang.String.<init>(String.java:207)
	at java.lang.String.substring(String.java:1969)
	at scala.collection.immutable.StringOps$.slice$extension(StringOps.scala:44)
	at org.broadinstitute.pilon.Assembler$$anonfun$addToPileups$1.apply$mcVI$sp(Assembler.scala:82)
	at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:160)
	at org.broadinstitute.pilon.Assembler.addToPileups(Assembler.scala:81)
	at org.broadinstitute.pilon.Assembler.addRead(Assembler.scala:70)
	at org.broadinstitute.pilon.Assembler$$anonfun$addReads$1.apply(Assembler.scala:48)
	at org.broadinstitute.pilon.Assembler$$anonfun$addReads$1.apply(Assembler.scala:48)
	at scala.collection.immutable.List.foreach(List.scala:381)
	at org.broadinstitute.pilon.Assembler.addReads(Assembler.scala:48)
	at org.broadinstitute.pilon.GapFiller.assembleIntoBreak(GapFiller.scala:124)
	at org.broadinstitute.pilon.GapFiller.assembleAcrossBreak(GapFiller.scala:52)
	at org.broadinstitute.pilon.GapFiller.fixBreak(GapFiller.scala:45)
	at org.broadinstitute.pilon.GenomeRegion$$anonfun$identifyAndFixIssues$4.apply(GenomeRegion.scala:383)
	at org.broadinstitute.pilon.GenomeRegion$$anonfun$identifyAndFixIssues$4.apply(GenomeRegion.scala:381)
	at scala.collection.immutable.List.foreach(List.scala:381)
	at org.broadinstitute.pilon.GenomeRegion.identifyAndFixIssues(GenomeRegion.scala:381)
	at org.broadinstitute.pilon.GenomeFile$$anonfun$processRegions$4.apply(GenomeFile.scala:119)
	at org.broadinstitute.pilon.GenomeFile$$anonfun$processRegions$4.apply(GenomeFile.scala:108)
	at scala.collection.Iterator$class.foreach(Iterator.scala:893)
	at scala.collection.AbstractIterator.foreach(Iterator.scala:1336)
	at scala.collection.parallel.ParIterableLike$Foreach.leaf(ParIterableLike.scala:972)
	at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply$mcV$sp(Tasks.scala:49)
	at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply(Tasks.scala:48)
	at scala.collection.parallel.Task$$anonfun$tryLeaf$1.apply(Tasks.scala:48)
	at scala.collection.parallel.Task$class.tryLeaf(Tasks.scala:51)
	at scala.collection.parallel.ParIterableLike$Foreach.tryLeaf(ParIterableLike.scala:969)
	at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask$class.compute(Tasks.scala:152)
	at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.compute(Tasks.scala:443)
	at scala.concurrent.forkjoin.RecursiveAction.exec(RecursiveAction.java:160)

Two things are strange here: The previous round of pilon didnt change anything, so the crashing round seems superfluous.
Also, unicycler does not seem to care and gives:

Assembly complete (2018-10-12 19:47:02)
Saving /data/mahlzeitlocal/sebastian/mycoplasma_bovis_asm/unicycler/15DD0163/assembly.gfa
Saving /data/mahlzeitlocal/sebastian/mycoplasma_bovis_asm/unicycler/15DD0163/assembly.fasta


real	269m19.594s
user	2189m14.516s
sys	10m13.884s
>>> Done.
>>> All done.

So I would expect it is indeed finished - still seems like a bug or some incorrect memory allocation for pilon.

@apredeus
Copy link

I've always changed the settings of memory that Pilon is allowed to use, otherwise it crashes quite often.

Just find pilon wrapper and change this line

default_jvm_mem_opts = ['-Xms512m', '-Xmx1g']

to

default_jvm_mem_opts = ['-Xms512m', '-Xmx8g']

or something appropriate for your system. I think 8 Gb should be enough for any bacterial genome.

@RaverJay
Copy link
Author

True, seems rather obvious that 1GiB is too low. Thanks!

For anyone seeing this and wondering about where to change the line:
The pilon executable (e.g. from conda) is just a python wrapper script to run the java program.

e.g. /home/<user>/miniconda3/bin/pilon

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants