resume fail with OutOfMemoryError while appending to tree log

[jheled]

I start with the largest possible -Xmx my 32bit cpu/java allows on my laptop. The trees file "only" 340M, and Remco says he successfully resumed larger files.

The file can be downloaded from
https://dl.dropboxusercontent.com/u/5675908/conus-al.tar.bz2

java -ea -Xmx2500m -Xss2024k -jar ... -seed 666 -resume conus-al.xml

...
Start likelihood: -30613.211722592572
Appending file conus-al.log
Sample posterior ESS(posterior) likelihood prior
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:2882)
at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100)
at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:390)
at java.lang.StringBuilder.append(StringBuilder.java:119)
at beast.core.Logger.openLogFile(Unknown Source)
at beast.core.Logger.init(Unknown Source)
at beast.core.MCMC.run(Unknown Source)
at beast.app.BeastMCMC.run(Unknown Source)
at beast.app.beastapp.BeastMain.(Unknown Source)
at beast.app.beastapp.BeastMain.main(Unknown Source)

[alexeid]

Are all the trees held in memory? That is obviously unnecessary for a resume.

Sent from my iPhone

On 1/02/2014, at 9:30 AM, Joseph Heled notifications@github.com wrote:

I start with the largest possible -Xmx my 32bit cpu/java allows on my laptop. The trees file "only" 340M, and Remco says he successfully resumed larger files.

The file can be downloaded from
https://dl.dropboxusercontent.com/u/5675908/conus-al.tar.bz2

java -ea -Xmx2500m -Xss2024k -jar ... -seed 666 -resume conus-al.xml

...
Start likelihood: -30613.211722592572
Appending file conus-al.log
Sample posterior ESS(posterior) likelihood prior
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:2882)
at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100)
at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:390)
at java.lang.StringBuilder.append(StringBuilder.java:119)
at beast.core.Logger.openLogFile(Unknown Source)
at beast.core.Logger.init(Unknown Source)
at beast.core.MCMC.run(Unknown Source)
at beast.app.BeastMCMC.run(Unknown Source)
at beast.app.beastapp.BeastMain.(Unknown Source)
at beast.app.beastapp.BeastMain.main(Unknown Source)

—
Reply to this email directly or view it on GitHub.

[jheled]

The code needs to extract the sample number and remove the last 'End;' line, so it seems it can write out the new log file while reading (to a new term file) and do the rename at the end. Since this is the obvious way I assume there was a reason why the original code read all the trees in, and would rather wait for Remco to have a look.

[rbouckaert]

Using -Xmx8g instead of -Xmx2500m solves this problem.
Also, fixed the code to keep tree not in memory.