whole genome example bcbio.variation problem

[bogklug]

Hi,

I'm trying to run the whole genome example pipeline and I'm getting the following error. Also, is there an easy way to make bcbio continue working from the place it failed?

Command failing:
java -Xms750m -Xmx2500m -Djava.io.tmpdir=/work/bcbio/bcbio_example_1/work/validate/NA12878/gatk/tmp -jar /bcbio/tools/share/java/bcbio_variation/bcbio.variation-0.1.1-standalone.jar variant-compare /work/bcbio/bcbio_example_1/work/validate/NA12878/gatk/config/validate.yaml


Error info:
org.broadinstitute.sting.utils.exceptions.UserException: Concordance Metrics is currently only implemented for DIPLOID genotypes, found eval ploidy: 0, comp ploidy: 1
at org.broadinstitute.sting.gatk.walkers.variantutils.ConcordanceMetrics.update(ConcordanceMetrics.java:130)
at org.broadinstitute.sting.gatk.walkers.variantutils.GenotypeConcordance.reduce(GenotypeConcordance.java:318)
at org.broadinstitute.sting.gatk.walkers.variantutils.GenotypeConcordance.reduce(GenotypeConcordance.java:133)
at org.broadinstitute.sting.gatk.traversals.TraverseLociNano$TraverseLociReduce.apply(TraverseLociNano.java:291)
at org.broadinstitute.sting.gatk.traversals.TraverseLociNano$TraverseLociReduce.apply(TraverseLociNano.java:280)
at org.broadinstitute.sting.utils.nanoScheduler.NanoScheduler.executeSingleThreaded(NanoScheduler.java:279)
at org.broadinstitute.sting.utils.nanoScheduler.NanoScheduler.execute(NanoScheduler.java:245)
at org.broadinstitute.sting.gatk.traversals.TraverseLociNano.traverse(TraverseLociNano.java:144)
at org.broadinstitute.sting.gatk.traversals.TraverseLociNano.traverse(TraverseLociNano.java:92)
at org.broadinstitute.sting.gatk.traversals.TraverseLociNano.traverse(TraverseLociNano.java:48)
at org.broadinstitute.sting.gatk.executive.LinearMicroScheduler.execute(LinearMicroScheduler.java:99)
at org.broadinstitute.sting.gatk.GenomeAnalysisEngine.execute(GenomeAnalysisEngine.java:313)
at org.broadinstitute.sting.gatk.CommandLineExecutable.execute(CommandLineExecutable.java:113)
at org.broadinstitute.sting.commandline.CommandLineProgram.start(CommandLineProgram.java:245)
at org.broadinstitute.sting.commandline.CommandLineProgram.start(CommandLineProgram.java:152)
at bcbio.run.broad$run_gatk$fn__1094.invoke(broad.clj:34)
at bcbio.run.broad$run_gatk.invoke(broad.clj:31)
at bcbio.variation.evaluate$calc_variant_eval_metrics.doInvoke(evaluate.clj:22)
at clojure.lang.RestFn.invoke(RestFn.java:573)
at bcbio.variation.compare$compare_two_vcf_standard.invoke(compare.clj:161)
at bcbio.variation.compare$compare_two_vcf.invoke(compare.clj:183)
at bcbio.variation.compare$variant_comparison_from_config$iter__6966__6970$fn__6971$iter__6990__6994$fn__6995.invoke(compare.clj:253)
at clojure.lang.LazySeq.sval(LazySeq.java:42)
at clojure.lang.LazySeq.seq(LazySeq.java:60)
at clojure.lang.RT.seq(RT.java:484)
at clojure.core$seq.invoke(core.clj:133)
at clojure.core.protocols$seq_reduce.invoke(protocols.clj:30)
at clojure.core.protocols$fn__6026.invoke(protocols.clj:54)
at clojure.core.protocols$fn__5979$G__5974__5992.invoke(protocols.clj:13)
at clojure.core$reduce.invoke(core.clj:6177)
at bcbio.variation.multiple$prep_cmp_name_lookup.doInvoke(multiple.clj:39)
at clojure.lang.RestFn.invoke(RestFn.java:410)
at bcbio.variation.compare$finalize_comparisons.invoke(compare.clj:230)
at bcbio.variation.compare$variant_comparison_from_config$iter__6966__6970$fn__6971.invoke(compare.clj:254)
at clojure.lang.LazySeq.sval(LazySeq.java:42)
at clojure.lang.LazySeq.seq(LazySeq.java:60)
at clojure.lang.RT.seq(RT.java:484)
at clojure.core$seq.invoke(core.clj:133)
at clojure.core$tree_seq$walk__4647$fn__4648.invoke(core.clj:4475)
at clojure.lang.LazySeq.sval(LazySeq.java:42)
at clojure.lang.LazySeq.seq(LazySeq.java:60)
at clojure.lang.LazySeq.more(LazySeq.java:96)
at clojure.lang.RT.more(RT.java:607)
at clojure.core$rest.invoke(core.clj:73)
at clojure.core$flatten.invoke(core.clj:6478)
at bcbio.variation.compare$variant_comparison_from_config.invoke(compare.clj:251)
at bcbio.variation.compare$_main.invoke(compare.clj:271)
at clojure.lang.AFn.applyToHelper(AFn.java:161)
at clojure.lang.AFn.applyTo(AFn.java:151)
at clojure.core$apply.invoke(core.clj:617)
at bcbio.variation.core$_main.doInvoke(core.clj:35)
at clojure.lang.RestFn.applyTo(RestFn.java:137)
at bcbio.variation.core.main(Unknown Source)


[chapmanb]

Thanks for the report. What version of bcbio.variation is reported as installed in provenance/programs.txt? If it's 0.1.1, you just need an upgrade to the latest version with a fix for this problem:

bcbio_nextgen.py upgrade --tools

Hopefully that will fix things so the example runs to completion. When you restart the pipeline bcbio will automatically pick up right where it left off so need to do anything special other than re-run after upgrading. Hope this fixes the issue for you.

[bogklug]

Thank you for your quick reply. You are right - it is bcbio.variation 0.1.1. When trying to upgrade i get:

Error: Failed to connect to: https://api.github.com/legacy/issues/search/mxcl/homebrew/open/bedtools
SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed

Fatal error: local() encountered an error (return code 1) while executing 'export CC=${CC:-which gcc} && export CXX=${CXX:-which g++} && export PERL5LIB=/bcbio/tools/lib/perl5:${PERL5LIB} && /bcbio/tools/bin/brew upgrade --env=inherit bedtools'

[chapmanb]

It looks like something happened with the SSL certificate checking. Is this a persistent problem or does it resolve if you retry? If persistent, what operating system and version of ruby do you have? It's strange this would happen after you previously had a successful install but perhaps something changed. Googling around on the error it looks like the fix is to install certificates but the exact method depends on your operating system. Hope this helps to start diagnosing it.

[bogklug]

It still persists. I'm running an isolated bcbio install on Ubuntu 12.04.2 LTS. It is possible that the configuration has changed a bit since the initial install. The ruby version is ruby 1.8.7 (2011-06-30 patchlevel 352) [x86_64-linux].

If I do export SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt then the SSL error is gone and instead I just get: Error: 404 Not Found

Perhaps I should have posted a longer log fragment:

==> Upgrading bedtools
==> Downloading https://github.com/arq5x/bedtools2/archive/v2.18.0.tar.gz
Already downloaded: /home/bcbio/.cache/Homebrew/bedtools-2.18.0.tar.gz
==> make
obj/InputStreamMgr.o: In function `InputStreamMgr::detectBamOrBgzip(int&, int)':
InputStreamMgr.cpp:(.text+0x54c): undefined reference to `BamTools::BamReader::OpenStream(std::basic_istream<char, std::char_traits<char> >*)'
InputStreamMgr.cpp:(.text+0x6df): undefined reference to `BamTools::Internal::BgzfStream::OpenStream(std::basic_istream<char, std::char_traits<char> >*, BamTools::IBamIODevice::OpenMode)'
collect2: ld returned 1 exit status
make: *** [all] Error 1

READ THIS: https://github.com/mxcl/homebrew/wiki/troubleshooting
If reporting this issue please do so at (not mxcl/homebrew):
  https://github.com/homebrew/homebrew-science/issues

Error: 404 Not Found
Please report this bug:
    https://github.com/mxcl/homebrew/wiki/troubleshooting
/usr/lib/ruby/1.8/open-uri.rb:277:in `open_http'
/usr/lib/ruby/1.8/open-uri.rb:616:in `buffer_open'
/usr/lib/ruby/1.8/open-uri.rb:164:in `open_loop'
/usr/lib/ruby/1.8/open-uri.rb:162:in `catch'
/usr/lib/ruby/1.8/open-uri.rb:162:in `open_loop'
/usr/lib/ruby/1.8/open-uri.rb:132:in `open_uri'
/usr/lib/ruby/1.8/open-uri.rb:518:in `open'
/usr/lib/ruby/1.8/open-uri.rb:26:in `open'
/bcbio/tools/Library/Homebrew/utils.rb:265:in `open'
/bcbio/tools/Library/Homebrew/utils.rb:282:in `each_issue_matching'
/bcbio/tools/Library/Homebrew/utils.rb:294:in `issues_for_formula'
/bcbio/tools/Library/Homebrew/exceptions.rb:178:in `issues'
/bcbio/tools/Library/Homebrew/exceptions.rb:212:in `dump'
/bcbio/tools/Library/Homebrew/cmd/upgrade.rb:88:in `upgrade_formula'
/bcbio/tools/Library/Homebrew/cmd/upgrade.rb:45:in `upgrade'
/bcbio/tools/Library/Homebrew/cmd/upgrade.rb:45:in `each'
/bcbio/tools/Library/Homebrew/cmd/upgrade.rb:45:in `upgrade'
/bcbio/tools/Library/brew.rb:91:in `send'
/bcbio/tools/Library/brew.rb:91

Fatal error: local() encountered an error (return code 1) while executing 'export CC=${CC:-`which gcc`} && export CXX=${CXX:-`which g++`} && export PERL5LIB=/bcbio/tools/lib/perl5:${PERL5LIB} && /bcbio/tools/bin/brew upgrade --env=inherit bedtools'

Aborting.

[chapmanb]

Sorry about the problem. It looks like there is a build error with the latest bedtools on CentOS/RedHat systems. I reported upstream and pushed a fix to use the previous 2.17.0 version until this is resolved. If you remove the temporary build directory (rm -rf tmpbcbio-install) and re-update, it should install that version and keep running. Please let us know if you run into any other issues.