bad_alloc error #32

Closed
tjjjohnson opened this Issue Apr 20, 2012 · 12 comments

Comments

Projects
None yet
3 participants
@tjjjohnson

I'm trying to run freebayes for a single chromosome on a population of 25 animals but usually get the following error:

terminate called after throwing an instance of 'std::bad_alloc'
 what():  std::bad_alloc

this is the command I'm using:

freebayes  -f  reference.fa -0 -b merged-chr1.bam > chr1-snp.vcf

I'm running this with 20G of ram available. The input bam is 109G. It generally only seems to consume about 230m of ram.

It fails at various times and runs to completion sometimes.

any help would be much appreciated.

@ekg

This comment has been minimized.

Show comment Hide comment
@ekg

ekg May 16, 2012

Owner

Is it possible for you to provide a test case that recreates the issue? Does this still occur in the most current git HEAD?

Owner

ekg commented May 16, 2012

Is it possible for you to provide a test case that recreates the issue? Does this still occur in the most current git HEAD?

@ekg

This comment has been minimized.

Show comment Hide comment
@ekg

ekg Oct 2, 2013

Owner

Please let me know if this is still a problem in the current releases. Reopen if this is the case.

Owner

ekg commented Oct 2, 2013

Please let me know if this is still a problem in the current releases. Reopen if this is the case.

@ekg ekg closed this Oct 2, 2013

@sasignor

This comment has been minimized.

Show comment Hide comment
@sasignor

sasignor Mar 19, 2014

This actually just happened to me with the latest release. I would have thought nothing of it, except I am simultaneously running the same command on my computer (32G of RAM) and a cluster (128G of RAM) and it is the one on the cluster that crashed.

terminate called after throwing an instance of 'std::bad_alloc'
what(): std::bad_alloc
Aborted

The command being run in both instances is as follows:
./freebayes -p 40 -N -F .2 -C 5 -H -j --min-mapping-quality 15 --min-base-quality 30 --min-coverage 10 --fasta-reference reference.fa readgrouped1_rmdup.bam readgrouped2_rmdup.bam > all_readgrouped_filt.vcf

This actually just happened to me with the latest release. I would have thought nothing of it, except I am simultaneously running the same command on my computer (32G of RAM) and a cluster (128G of RAM) and it is the one on the cluster that crashed.

terminate called after throwing an instance of 'std::bad_alloc'
what(): std::bad_alloc
Aborted

The command being run in both instances is as follows:
./freebayes -p 40 -N -F .2 -C 5 -H -j --min-mapping-quality 15 --min-base-quality 30 --min-coverage 10 --fasta-reference reference.fa readgrouped1_rmdup.bam readgrouped2_rmdup.bam > all_readgrouped_filt.vcf

@sasignor

This comment has been minimized.

Show comment Hide comment
@sasignor

sasignor Mar 21, 2014

Changing the parameters does not change the error, I tried getting rid of -j and -H, changing -C and -p to less, and get the same error....

Changing the parameters does not change the error, I tried getting rid of -j and -H, changing -C and -p to less, and get the same error....

@ekg

This comment has been minimized.

Show comment Hide comment
@ekg

ekg Mar 21, 2014

Owner

Are you sure you are suffering from memory exhaustion? It would seem so
given the error, but I'm curious.

What kinds of depths do you see (e.g. via bamtools coverage -in input.bam)
in the region?

On Fri, Mar 21, 2014 at 2:14 PM, sasignor notifications@github.com wrote:

Changing the parameters does not change the error, I tried getting rid of
-j and -H, changing -C and -p to less, and get the same error....

Reply to this email directly or view it on GitHubhttps://github.com/ekg/freebayes/issues/32#issuecomment-38306183
.

Owner

ekg commented Mar 21, 2014

Are you sure you are suffering from memory exhaustion? It would seem so
given the error, but I'm curious.

What kinds of depths do you see (e.g. via bamtools coverage -in input.bam)
in the region?

On Fri, Mar 21, 2014 at 2:14 PM, sasignor notifications@github.com wrote:

Changing the parameters does not change the error, I tried getting rid of
-j and -H, changing -C and -p to less, and get the same error....

Reply to this email directly or view it on GitHubhttps://github.com/ekg/freebayes/issues/32#issuecomment-38306183
.

@sasignor

This comment has been minimized.

Show comment Hide comment
@sasignor

sasignor Mar 21, 2014

The average for a single .bam file is around 40x, and there are 13 files going into the final analysis. Of course there are many regions with much lower coverage, the max being around 2500.

The average for a single .bam file is around 40x, and there are 13 files going into the final analysis. Of course there are many regions with much lower coverage, the max being around 2500.

@ekg

This comment has been minimized.

Show comment Hide comment
@ekg

ekg Mar 21, 2014

Owner

Can you reproduce the error reliably? Maybe I could examine a test case?
On Mar 21, 2014 10:41 PM, "sasignor" notifications@github.com wrote:

The average for a single .bam file is around 40x, and there are 13 files
going into the final analysis. Of course there are many regions with much
lower coverage, the max being around 2500.

Reply to this email directly or view it on GitHubhttps://github.com/ekg/freebayes/issues/32#issuecomment-38327195
.

Owner

ekg commented Mar 21, 2014

Can you reproduce the error reliably? Maybe I could examine a test case?
On Mar 21, 2014 10:41 PM, "sasignor" notifications@github.com wrote:

The average for a single .bam file is around 40x, and there are 13 files
going into the final analysis. Of course there are many regions with much
lower coverage, the max being around 2500.

Reply to this email directly or view it on GitHubhttps://github.com/ekg/freebayes/issues/32#issuecomment-38327195
.

@sasignor

This comment has been minimized.

Show comment Hide comment
@sasignor

sasignor Mar 21, 2014

Yes, I can. Do you mean you would like my input file?

Yes, I can. Do you mean you would like my input file?

@ekg

This comment has been minimized.

Show comment Hide comment
@ekg

ekg Mar 22, 2014

Owner

If when running on a particular region (small hopefully) you can cause this
error deterministically, the input files will be enough information to
figure out and resolve the error. Do you think this is a possibility?
On Mar 22, 2014 12:58 AM, "sasignor" notifications@github.com wrote:

Yes, I can. Do you mean you would like my input file?

Reply to this email directly or view it on GitHubhttps://github.com/ekg/freebayes/issues/32#issuecomment-38336502
.

Owner

ekg commented Mar 22, 2014

If when running on a particular region (small hopefully) you can cause this
error deterministically, the input files will be enough information to
figure out and resolve the error. Do you think this is a possibility?
On Mar 22, 2014 12:58 AM, "sasignor" notifications@github.com wrote:

Yes, I can. Do you mean you would like my input file?

Reply to this email directly or view it on GitHubhttps://github.com/ekg/freebayes/issues/32#issuecomment-38336502
.

@sasignor

This comment has been minimized.

Show comment Hide comment
@sasignor

sasignor Mar 23, 2014

Sure, but I would rather not post it publicly.

Cheers, Sarah
Postdoctoral Research Fellow
University of Southern California
Ray R. Irani Hall Rm 316
1050 Child's Way
P.O. 100730064
Los Angeles, CA 90089

On Sat, Mar 22, 2014 at 12:44 AM, Erik Garrison notifications@github.comwrote:

If when running on a particular region (small hopefully) you can cause this
error deterministically, the input files will be enough information to
figure out and resolve the error. Do you think this is a possibility?
On Mar 22, 2014 12:58 AM, "sasignor" notifications@github.com wrote:

Yes, I can. Do you mean you would like my input file?

Reply to this email directly or view it on GitHub<
https://github.com/ekg/freebayes/issues/32#issuecomment-38336502>

.

Reply to this email directly or view it on GitHubhttps://github.com/ekg/freebayes/issues/32#issuecomment-38345412
.

Sure, but I would rather not post it publicly.

Cheers, Sarah
Postdoctoral Research Fellow
University of Southern California
Ray R. Irani Hall Rm 316
1050 Child's Way
P.O. 100730064
Los Angeles, CA 90089

On Sat, Mar 22, 2014 at 12:44 AM, Erik Garrison notifications@github.comwrote:

If when running on a particular region (small hopefully) you can cause this
error deterministically, the input files will be enough information to
figure out and resolve the error. Do you think this is a possibility?
On Mar 22, 2014 12:58 AM, "sasignor" notifications@github.com wrote:

Yes, I can. Do you mean you would like my input file?

Reply to this email directly or view it on GitHub<
https://github.com/ekg/freebayes/issues/32#issuecomment-38336502>

.

Reply to this email directly or view it on GitHubhttps://github.com/ekg/freebayes/issues/32#issuecomment-38345412
.

@ekg

This comment has been minimized.

Show comment Hide comment
@ekg

ekg Mar 23, 2014

Owner

Of course! If you would like simply email me privately. My address is in
the documentation and help text.
On Mar 23, 2014 8:24 PM, "sasignor" notifications@github.com wrote:

Sure, but I would rather not post it publicly.

Cheers, Sarah
Postdoctoral Research Fellow
University of Southern California
Ray R. Irani Hall Rm 316
1050 Child's Way
P.O. 100730064
Los Angeles, CA 90089

On Sat, Mar 22, 2014 at 12:44 AM, Erik Garrison <notifications@github.com

wrote:

If when running on a particular region (small hopefully) you can cause
this
error deterministically, the input files will be enough information to
figure out and resolve the error. Do you think this is a possibility?
On Mar 22, 2014 12:58 AM, "sasignor" notifications@github.com wrote:

Yes, I can. Do you mean you would like my input file?

Reply to this email directly or view it on GitHub<
https://github.com/ekg/freebayes/issues/32#issuecomment-38336502>

.

Reply to this email directly or view it on GitHub<
https://github.com/ekg/freebayes/issues/32#issuecomment-38345412>
.

Reply to this email directly or view it on GitHubhttps://github.com/ekg/freebayes/issues/32#issuecomment-38394030
.

Owner

ekg commented Mar 23, 2014

Of course! If you would like simply email me privately. My address is in
the documentation and help text.
On Mar 23, 2014 8:24 PM, "sasignor" notifications@github.com wrote:

Sure, but I would rather not post it publicly.

Cheers, Sarah
Postdoctoral Research Fellow
University of Southern California
Ray R. Irani Hall Rm 316
1050 Child's Way
P.O. 100730064
Los Angeles, CA 90089

On Sat, Mar 22, 2014 at 12:44 AM, Erik Garrison <notifications@github.com

wrote:

If when running on a particular region (small hopefully) you can cause
this
error deterministically, the input files will be enough information to
figure out and resolve the error. Do you think this is a possibility?
On Mar 22, 2014 12:58 AM, "sasignor" notifications@github.com wrote:

Yes, I can. Do you mean you would like my input file?

Reply to this email directly or view it on GitHub<
https://github.com/ekg/freebayes/issues/32#issuecomment-38336502>

.

Reply to this email directly or view it on GitHub<
https://github.com/ekg/freebayes/issues/32#issuecomment-38345412>
.

Reply to this email directly or view it on GitHubhttps://github.com/ekg/freebayes/issues/32#issuecomment-38394030
.

@ekg

This comment has been minimized.

Show comment Hide comment
@ekg

ekg Apr 16, 2014

Owner

Hi,

I've tested the case that Sarah sent, and find that the problem seems to be
the combination of --ploidy 40 and loci with high numbers of possible
variants.

To resolve this, force freebayes to only consider biallelics or triallelics:

--use-best-n-alleles 2 # biallelics considered
--use-best-n-alleles 3 # biallelics and triallelics considered

Note that pooled calling with 40 samples in a pool is very
memory-intensive. At a certain pool size it may make more sense to simply
call with --ploidy 1 and --pooled-continuous, then process the counts as
pool frequencies (as provided by AO and RO in the per-sample output).

Erik

On Sun, Mar 23, 2014 at 8:24 PM, sasignor notifications@github.com wrote:

Sure, but I would rather not post it publicly.

Cheers, Sarah
Postdoctoral Research Fellow
University of Southern California
Ray R. Irani Hall Rm 316
1050 Child's Way
P.O. 100730064
Los Angeles, CA 90089

On Sat, Mar 22, 2014 at 12:44 AM, Erik Garrison <notifications@github.com

wrote:

If when running on a particular region (small hopefully) you can cause
this
error deterministically, the input files will be enough information to
figure out and resolve the error. Do you think this is a possibility?
On Mar 22, 2014 12:58 AM, "sasignor" notifications@github.com wrote:

Yes, I can. Do you mean you would like my input file?

Reply to this email directly or view it on GitHub<
https://github.com/ekg/freebayes/issues/32#issuecomment-38336502>

.

Reply to this email directly or view it on GitHub<
https://github.com/ekg/freebayes/issues/32#issuecomment-38345412>
.


Reply to this email directly or view it on GitHubhttps://github.com/ekg/freebayes/issues/32#issuecomment-38394030
.

Owner

ekg commented Apr 16, 2014

Hi,

I've tested the case that Sarah sent, and find that the problem seems to be
the combination of --ploidy 40 and loci with high numbers of possible
variants.

To resolve this, force freebayes to only consider biallelics or triallelics:

--use-best-n-alleles 2 # biallelics considered
--use-best-n-alleles 3 # biallelics and triallelics considered

Note that pooled calling with 40 samples in a pool is very
memory-intensive. At a certain pool size it may make more sense to simply
call with --ploidy 1 and --pooled-continuous, then process the counts as
pool frequencies (as provided by AO and RO in the per-sample output).

Erik

On Sun, Mar 23, 2014 at 8:24 PM, sasignor notifications@github.com wrote:

Sure, but I would rather not post it publicly.

Cheers, Sarah
Postdoctoral Research Fellow
University of Southern California
Ray R. Irani Hall Rm 316
1050 Child's Way
P.O. 100730064
Los Angeles, CA 90089

On Sat, Mar 22, 2014 at 12:44 AM, Erik Garrison <notifications@github.com

wrote:

If when running on a particular region (small hopefully) you can cause
this
error deterministically, the input files will be enough information to
figure out and resolve the error. Do you think this is a possibility?
On Mar 22, 2014 12:58 AM, "sasignor" notifications@github.com wrote:

Yes, I can. Do you mean you would like my input file?

Reply to this email directly or view it on GitHub<
https://github.com/ekg/freebayes/issues/32#issuecomment-38336502>

.

Reply to this email directly or view it on GitHub<
https://github.com/ekg/freebayes/issues/32#issuecomment-38345412>
.


Reply to this email directly or view it on GitHubhttps://github.com/ekg/freebayes/issues/32#issuecomment-38394030
.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment