gemini error

[MengNiu]

Hi,

I updated the gemini as I saw in the related posts, but still getting the error as following:

Thanks for the help!

---

Traceback (most recent call last):
File "/home/mn/local/share/bcbio/anaconda/bin/gemini", line 6, in
gemini.gemini_main.main()
File "/home/mn/local/share/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1227, in main
args.func(parser, args)
File "/home/mn/local/share/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 198, in load_fn
gemini_load.load(parser, args)
File "/home/mn/local/share/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 48, in load
load_multicore(args)
File "/home/mn/local/share/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 92, in load_multicore
chunks = load_chunks_multicore(grabix_file, args)
File "/home/mn/local/share/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 258, in load_chunks_multicore
wait_until_finished(procs)
File "/home/mn/local/share/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 349, in wait_until_finished
raise ValueError("Processing failed on GEMINI chunk load")
ValueError: Processing failed on GEMINI chunk load
Uncaught exception occurred
Traceback (most recent call last):
File "/home/mn/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py", line 21, in run
_do_run(cmd, checks, log_stdout)
File "/home/mn/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py", line 95, in _do_run
raise subprocess.CalledProcessError(exitcode, error_msg)

CalledProcessError: Command 'set -o pipefail; /home/mn/local/share/bcbio/anaconda/bin/gemini load --passonly --skip-gerp-bp -v /home/mn/test/work/gemini/test11-varscan-decompose-effects.vcf.gz

Traceback (most recent call last):
File "/home/mn/local/share/bcbio/anaconda/bin/gemini", line 6, in
gemini.gemini_main.main()
File "/home/mn/local/share/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1227, in main
args.func(parser, args)
File "/home/mn/local/share/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 198, in load_fn
gemini_load.load(parser, args)
File "/home/mn/local/share/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 48, in load
load_multicore(args)
File "/home/mn/local/share/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 92, in load_multicore
chunks = load_chunks_multicore(grabix_file, args)
File "/home/mn/local/share/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 258, in load_chunks_multicore
wait_until_finished(procs)
File "/home/mn/local/share/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load.py", line 349, in wait_until_finished
raise ValueError("Processing failed on GEMINI chunk load")
ValueError: Processing failed on GEMINI chunk load
' returned non-zero exit status 1

[chapmanb]

Thanks for the report and sorry about the issues. It looks like there is not enough information in the traceback you posted to diagnose the underlying issue. GEMINI is failing for some reason but the causative error must be higher up in the log file. You can try diagnosing by running the failing command:

 /home/mn/local/share/bcbio/anaconda/bin/gemini load --passonly --skip-gerp-bp -v /home/mn/test/work/gemini/test11-varscan-decompose-effects.vcf.gz test.db

This will hopefully provide some details around why GEMINI failed to help with debugging.

[MengNiu]

Hi Brad,

Thanks for the reply! The output of the command listed is:

---

CADD scores are being loaded (to skip use:--skip-cadd).
pid 55207: 4462 variants processed.
pid 55207: 126902 skipped due to having the FILTER field set.
storing version, header, etc.
storing gene-detailed
storing gene-summary
updating gene-table
building indices

---

The very first error in bcbio-nextgen.log is (it was shown on the job on node3(ica3)):

[2016-08-26T15:40Z] ica3: Uncaught exception occurred
Traceback (most recent call last):
File "/home/mn/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py", line 21, in run
_do_run(cmd, checks, log_stdout)
File "/home/mn/local/share/bcbio/anaconda/lib/python2.7/site-packages/bcbio/provenance/do.py", line 95, in _do_run
raise subprocess.CalledProcessError(exitcode, error_msg)
CalledProcessError: Command 'set -o pipefail; /home/mn/local/share/bcbio/anaconda/bin/gemini load --passonly --skip-gerp-bp -v /home/mn/Ccw/test/work/gemini/5961-ensemble-decompose-effects.vcf.gz -t snpEff --cores 16 --tempdir /home/mn/Ccw/test/work/gemini/tx/tmphiAnup /home/mn/Ccw/test/work/gemini/tx/tmphiAnup/5961-ensemble.db
File "/home/mn/local/share/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load_chunk.py", line 201, in populate_from_vcf
(variant, variant_impacts, extra_fields) = self._prepare_variation(var, anno_keys)
File "/home/mn/local/share/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load_chunk.py", line 489, in _prepare_variation
gt_bases = var.gt_bases
File "cyvcf2/cyvcf2.pyx", line 725, in cyvcf2.cyvcf2.Variant.gt_bases.get (cyvcf2/cyvcf2.c:19337)
IndexError: list index out of range
Traceback (most recent call last):
File "/home/mn/local/bin/gemini", line 6, in
gemini.gemini_main.main()
File "/home/mn/local/share/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1227, in main
args.func(parser, args)
File "/home/mn/local/share/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 298, in loadchunk_fn
gemini_load_chunk.load(parser, args)
File "/home/mn/local/share/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load_chunk.py", line 843, in load
gemini_loader.populate_from_vcf()
File "/home/mn/local/share/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load_chunk.py", line 201, in populate_from_vcf
(variant, variant_impacts, extra_fields) = self._prepare_variation(var, anno_keys)
File "/home/mn/local/share/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load_chunk.py", line 489, in _prepare_variation
gt_bases = var.gt_bases
File "cyvcf2/cyvcf2.pyx", line 725, in cyvcf2.cyvcf2.Variant.gt_bases.get (cyvcf2/cyvcf2.c:19337)
IndexError: list index out of range

The first error on the test1.log(where test1.slurm is the slurm file), it showed the job was on node2(ica2):

[2016-08-26T16:21Z] ica2: File "/home/mn/local/bin/gemini", line 6, in
[2016-08-26T16:21Z] ica2: gemini.gemini_main.main()
[2016-08-26T16:21Z] ica2: File "/home/mn/local/share/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 1227, in main
[2016-08-26T16:21Z] ica2: args.func(parser, args)
[2016-08-26T16:21Z] ica2: File "/home/mn/local/share/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_main.py", line 298, in loadchunk_fn
[2016-08-26T16:21Z] ica2: gemini_load_chunk.load(parser, args)
[2016-08-26T16:21Z] ica2: File "/home/mn/local/share/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load_chunk.py", line 843, in load
[2016-08-26T16:21Z] ica2: gemini_loader.populate_from_vcf()
[2016-08-26T16:21Z] ica2: File "/home/mn/local/share/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load_chunk.py", line 201, in populate_from_vcf
[2016-08-26T16:21Z] ica2: (variant, variant_impacts, extra_fields) = self._prepare_variation(var, anno_keys)
[2016-08-26T16:21Z] ica2: File "/home/mn/local/share/bcbio/anaconda/lib/python2.7/site-packages/gemini/gemini_load_chunk.py", line 525, in _prepare_variation
[2016-08-26T16:21Z] ica2: impact_so=impact.so, impact_severity=impact.effect_severity,
[2016-08-26T16:21Z] ica2: File "/home/mn/local/share/bcbio/anaconda/lib/python2.7/site-packages/geneimpacts/effect.py", line 331, in effect_severity
[2016-08-26T16:21Z] ica2: return self.impact_severity
[2016-08-26T16:21Z] ica2: File "/home/mn/local/share/bcbio/anaconda/lib/python2.7/site-packages/geneimpacts/effect.py", line 365, in impact_severity
[2016-08-26T16:21Z] ica2: return ['xxx', 'LOW', 'MED', 'HIGH'][self.severity]
[2016-08-26T16:21Z] ica2: File "/home/mn/local/share/bcbio/anaconda/lib/python2.7/site-packages/geneimpacts/effect.py", line 356, in severity
[2016-08-26T16:21Z] ica2: v = max(lookup[sev[csq]] for csq in self.consequences)
[2016-08-26T16:21Z] ica2: File "/home/mn/local/share/bcbio/anaconda/lib/python2.7/site-packages/geneimpacts/effect.py", line 356, in
[2016-08-26T16:21Z] ica2: v = max(lookup[sev[csq]] for csq in self.consequences)
[2016-08-26T16:21Z] ica2: KeyError: 'gene_fusion'

[chapmanb]

Thanks for the details. To fix these, could you try doing:

bcbio_conda install -c bioconda geneimpacts cyvcf2

That should get you geneimpacts 0.1.4_1 and cyvcf2 0.5.3_0. I just pushed a fix for the geneimpacts problem and the cyvcf2 problem was fixed a while back (brentp/cyvcf2#14) but you might still have an old version.

After this you should be able to re-start your analysis from where it finished and hopefully things will finish successfully.

[MengNiu]

Hi Brad,

Thanks for the fix, it finished successfully!

Meng