New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

error with gbk #154

Closed
aemiol opened this Issue Apr 25, 2018 · 23 comments

Comments

Projects
None yet
10 participants
@aemiol

aemiol commented Apr 25, 2018

When I provide a gbk reference input, I get the following error below. This did not occur using a fasta reference.
Thanks.

java.lang.RuntimeException: Error parsing property 'ref.AE015924.codonTable'. No such codon table 'Bacterial_and_Plant_Plastid'
at org.snpeff.snpEffect.Config.createCodonTables(Config.java:169)
at org.snpeff.snpEffect.Config.readConfig(Config.java:650)
at org.snpeff.snpEffect.Config.init(Config.java:480)
at org.snpeff.snpEffect.Config.(Config.java:117)
at org.snpeff.SnpEff.loadConfig(SnpEff.java:451)
at org.snpeff.snpEffect.commandLine.SnpEffCmdBuild.run(SnpEffCmdBuild.java:364)
at org.snpeff.SnpEff.run(SnpEff.java:1183)
at org.snpeff.SnpEff.main(SnpEff.java:162)

@tseemann tseemann self-assigned this Apr 26, 2018

@tseemann

This comment has been minimized.

Owner

tseemann commented Apr 26, 2018

Seems similar to #153 issue.

What version of snpeff -version are you using?

Can you link to , or send me, the Genbank file?

@tseemann tseemann added the question label Apr 26, 2018

@aemiol

This comment has been minimized.

aemiol commented Apr 26, 2018

I installed snippy using conda and currently have snpeff 4.3. Snippy fails irrespective of the genbank file used. It even fails with the example.gbk file

@tseemann

This comment has been minimized.

Owner

tseemann commented Apr 27, 2018

Snippy will not work with some older genbank files that have lots of RBS features.
You have to grep them out first.
I assume it works with .fasta files ok?

@Slugger70

This comment has been minimized.

Slugger70 commented Apr 27, 2018

Galaxy snippy based on the bioconda passes all of it's tests with test gbk file etc. Which has no RBS features.

@sbridel

This comment has been minimized.

sbridel commented Apr 27, 2018

well, maybe you should provide more details about making proper GBK file.

My GBK file isn't working too and I have no idea how to clean it.

EDIT.

My GBK file contains 0 RBS features....

But I find something in snpeff documentation:


Building a database
In order to build a database for a new genome, you need to:

Most people do NOT need to build a database, and can safely use a pre-built one. So unless you are working with a rare genome you most likely don't need to do it either. 

or

Configuring a new genome

In order to tell SnpEff that there is a new genome available, you must update SnpEff's configuration file snpEff.config.
You must add a new genome entry to snpEff.config.
If your genome, or a chromosome, uses non-standard codon tables you must update snpEff.config accordingly. A typical case is when you use mitochondrial DNA. Then you specify that chromosome 'MT' uses codon.Invertebrate_Mitochondrial codon table. Another common case is when you are adding a bacterial genome, then you specify that the codon table is Bacterial_and_Plant_Plastid. 
@aemiol

This comment has been minimized.

aemiol commented Apr 27, 2018

My genbank file has no RBS features as well, yet getting error with snpEff

@tseemann

This comment has been minimized.

Owner

tseemann commented Apr 28, 2018

@sbridel yes, i do all those steps within Snippy to allow it to utilise the users reference genome. i suspect somehting has changed with snpeff

@sbridel and @aemiol can you pleas either

  1. paste your snps.log file here, or
  2. at least run run these commands and also tell me your reference file?
% snippy --version
snippy 3.x

% which snpEff
/home/linuxbrew/.linuxbrew/bin/snpEff

% snpEff -version
SnpEff  4.3t    2017-11-24
@aemiol

This comment has been minimized.

aemiol commented Apr 30, 2018

cd /projects/emiola/JG/contigs

/home/emiola/anaconda2/bin/snippy --outdir mut1 --ref sequence.gb --ctgs 14190_pginvivalisParent.fasta

samtools faidx reference/ref.fa

bwa index reference/ref.fa

[bwa_index] Pack FASTA... 0.05 sec
[bwa_index] Construct BWT for the packed sequence...
[bwa_index] 0.79 seconds elapse.
[bwa_index] Update BWT... 0.01 sec
[bwa_index] Pack forward-only FASTA... 0.01 sec
[bwa_index] Construct SA from BWT and Occ... 0.29 sec
[main] Version: 0.7.17-r1188
[main] CMD: bwa index reference/ref.fa
[main] Real time: 1.256 sec; CPU: 1.162 sec

mkdir reference/genomes && cp -f reference/ref.fa reference/genomes/ref.fa

mkdir reference/ref && gzip -c reference/ref.gff > reference/ref/genes.gff.gz

snpEff build -c reference/snpeff.config -dataDir . -gff3 ref

java.lang.RuntimeException: Error parsing property 'ref.AE015924.codonTable'. No such codon table 'Bacterial_and_Plant_Plastid'
at org.snpeff.snpEffect.Config.createCodonTables(Config.java:169)
at org.snpeff.snpEffect.Config.readConfig(Config.java:650)
at org.snpeff.snpEffect.Config.init(Config.java:480)
at org.snpeff.snpEffect.Config.(Config.java:117)
at org.snpeff.SnpEff.loadConfig(SnpEff.java:451)
at org.snpeff.snpEffect.commandLine.SnpEffCmdBuild.run(SnpEffCmdBuild.java:364)
at org.snpeff.SnpEff.run(SnpEff.java:1183)
at org.snpeff.SnpEff.main(SnpEff.java:162)

@moks-micro

This comment has been minimized.

moks-micro commented Apr 30, 2018

Hi,

I am having the same issue when using a genbank file. I am using a genbank file downloaded from NCBI (genome accession number is NZ_CP015278). Is there a fix to this ?

I am using snippy 4.0 and snpEff 4.3 version

My snp.log is :

cd /home/sim/Documents/snippy

/home/sim/miniconda2/bin/snippy --outdir test1 --ref DSM44623.gb --R1 NGS1_forward_paired.fq.gz --R2 NGS1_reverse_paired.fq.gz

samtools faidx reference/ref.fa

bwa index reference/ref.fa

[bwa_index] Pack FASTA... 0.00 sec
[bwa_index] Construct BWT for the packed sequence...
[bwa_index] 0.00 seconds elapse.
[bwa_index] Update BWT... 0.00 sec
[bwa_index] Pack forward-only FASTA... 0.00 sec
[bwa_index] Construct SA from BWT and Occ... 0.00 sec
[main] Version: 0.7.17-r1188
[main] CMD: bwa index reference/ref.fa
[main] Real time: 0.030 sec; CPU: 0.000 sec

mkdir reference/genomes && cp -f reference/ref.fa reference/genomes/ref.fa

mkdir reference/ref && gzip -c reference/ref.gff > reference/ref/genes.gff.gz

snpEff build -c reference/snpeff.config -dataDir . -gff3 ref

java.lang.RuntimeException: Error parsing property 'ref.NZ_CP015278.codonTable'. No such codon table 'Bacterial_and_Plant_Plastid'
at org.snpeff.snpEffect.Config.createCodonTables(Config.java:165)
at org.snpeff.snpEffect.Config.readConfig(Config.java:626)
at org.snpeff.snpEffect.Config.init(Config.java:468)
at org.snpeff.snpEffect.Config.(Config.java:114)
at org.snpeff.SnpEff.loadConfig(SnpEff.java:292)
at org.snpeff.snpEffect.commandLine.SnpEffCmdBuild.run(SnpEffCmdBuild.java:341)
at org.snpeff.SnpEff.run(SnpEff.java:1009)
at org.snpeff.SnpEff.main(SnpEff.java:155)

@cizydorczyk

This comment has been minimized.

cizydorczyk commented May 1, 2018

Hi,

I'm also getting the same error as above, with the following reference: NC_002516

snippy --version
snippy 4.0-dev

which snpEff
/home/conrad/anaconda2/bin/snpEff

snpEff -version
4.3

Note that I installed Snippy using anaconda.

The genbank file I am using was last updated Feb 2018, and I have used it in other applications (eg. Prokka) and it has always worked fine.

SNP log:

### cd /home/conrad/Data/all_pseudo

### /home/conrad/anaconda2/bin/snippy --cpus 8 --outdir K6_snippy/6 --ref genome_annotations/K2_pseudomonas_aeruginosa_pao1_apr24_2018.gbk --R1 fastq_files/all_good_fastq/6_1.fq.gz --R2 fastq_files/all_good_fastq/6_2.fq.gz

### samtools faidx reference/ref.fa


### bwa index reference/ref.fa

[bwa_index] Pack FASTA... 0.05 sec
[bwa_index] Construct BWT for the packed sequence...
[bwa_index] 2.07 seconds elapse.
[bwa_index] Update BWT... 0.06 sec
[bwa_index] Pack forward-only FASTA... 0.04 sec
[bwa_index] Construct SA from BWT and Occ... 0.62 sec
[main] Version: 0.7.17-r1188
[main] CMD: bwa index reference/ref.fa
[main] Real time: 3.821 sec; CPU: 2.840 sec

### mkdir reference/genomes && cp -f reference/ref.fa reference/genomes/ref.fa


### mkdir reference/ref && gzip -c reference/ref.gff > reference/ref/genes.gff.gz


### snpEff build -c reference/snpeff.config -dataDir . -gff3 ref

java.lang.RuntimeException: Error parsing property 'ref.NC_002516.codonTable'. No such codon table 'Bacterial_and_Plant_Plastid'
	at org.snpeff.snpEffect.Config.createCodonTables(Config.java:165)
	at org.snpeff.snpEffect.Config.readConfig(Config.java:626)
	at org.snpeff.snpEffect.Config.init(Config.java:468)
	at org.snpeff.snpEffect.Config.<init>(Config.java:114)
	at org.snpeff.SnpEff.loadConfig(SnpEff.java:292)
	at org.snpeff.snpEffect.commandLine.SnpEffCmdBuild.run(SnpEffCmdBuild.java:341)
	at org.snpeff.SnpEff.run(SnpEff.java:1009)
	at org.snpeff.SnpEff.main(SnpEff.java:155)
@stephenturner

This comment has been minimized.

stephenturner commented May 1, 2018

Found this just now, having the same issue.

### snpEff build -c reference/snpeff.config -dataDir . -gff3 ref

java.lang.RuntimeException: Error parsing property 'ref.NC_002944.codonTable'. No such codon table 'Bacterial_and_Plant_Plastid'
        at org.snpeff.snpEffect.Config.createCodonTables(Config.java:169)
        at org.snpeff.snpEffect.Config.readConfig(Config.java:650)
        at org.snpeff.snpEffect.Config.init(Config.java:480)
        at org.snpeff.snpEffect.Config.<init>(Config.java:117)
        at org.snpeff.SnpEff.loadConfig(SnpEff.java:451)
        at org.snpeff.snpEffect.commandLine.SnpEffCmdBuild.run(SnpEffCmdBuild.java:364)
        at org.snpeff.SnpEff.run(SnpEff.java:1183)
        at org.snpeff.SnpEff.main(SnpEff.java:162)
$ snippy --version
snippy 4.0-dev
$ snpEff -version
SnpEff  4.3t    2017-11-24
@Walwa

This comment has been minimized.

Walwa commented May 4, 2018

Same issue on my machine with this set up
$ snippy --version
snippy 4.0-dev
$ which snpEff
/conda/envs/genew1/bin/snpEff
$ snpEff -version
4.3
I get the same error message I had wondered if it was because I am looking at a Mycoplasma spp. but not it looks like its something in the software.

@Slugger70

This comment has been minimized.

Slugger70 commented May 4, 2018

Hi all,

The snippy 4.0-dev that conda uses as default was accidentally built from a pre-release and so is not ready for use. The fix is to use snippy version 3.2.

If you have snippy 4.0-dev installed from conda, then you need to remove it first and install version 3.2:

conda remove snippy
conda install -c bioconda -c conda-forge snippy=3.2

Hopefully, this will also help with the snpEff errors.

@tseemann

This comment has been minimized.

Owner

tseemann commented May 5, 2018

A proper 4.0 release is coming soon. until now the -dev1 -dev2 etc are for masochists.

(We are using in research, but it appears snpEff is causing issues for many)

@stephenturner

This comment has been minimized.

stephenturner commented May 7, 2018

Rolled back to version 3.2 and I'm still running into errors trying to use the gbk file instead of the fasta.

The snippy logs:

[07:41:52] Extracting FASTA and GFF from reference.
Use of uninitialized value in uc at /home/sdt5z/miniconda3/envs/mavium/bin/snippy line 159, <GEN0> line 101483.
[07:41:54] Wrote 1 sequences to ref.fa
[07:41:54] Wrote 4570 features to ref.gff
[07:41:54] Creating reference/snpeff.config
[07:41:54] Freebayes will process 96 chunks of 1000 bp, 24 chunks at a time.
[07:41:54] Using BAM RG (Read Group) ID: snps
[07:41:54] Running: samtools faidx reference/ref.fa 2>> snps.log
[07:41:54] Running: bwa index reference/ref.fa 2>> snps.log
[07:41:54] Running: mkdir reference/genomes && cp -f reference/ref.fa reference/genomes/ref.fa 2>> snps.log
[07:41:54] Running: mkdir reference/ref && bgzip -c reference/ref.gff > reference/ref/genes.gff.gz 2>> snps.log
[07:41:54] Running: snpEff build -c reference/snpeff.config -dataDir . -gff3 ref 2>> snps.log
[07:41:57] Error running command, check snippy-DW1/snps.log

And the snps.log file

### snpEff build -c reference/snpeff.config -dataDir . -gff3 ref

WARNING: All frames are zero! This seems rather odd, please check that 'frame' information in your 'genes' file is accurate.
java.lang.RuntimeException: Error reading file '/nv/vol184/uvabx/projects/houpt/darwin-mavium/pilot/snippy/snippy-DW1/reference/./ref/genes.gff'
java.lang.RuntimeException: FATAL ERROR: Most Exons do not have sequences!


. File '/path/snippy/snippy-DW1/reference/./ref/genes.gff' line 4573
        'NC_002944      snippy  CDS     4829033 4829176 .       -       0       ID=MAP_RS23120;codon_start=1;db_xref=GeneID:31477900;inference=COORDINATES: similar to AA sequence:RefSeq:WP_005263481.1;locus_tag=MAP_RS23120;note=Derived by automated computational analysis using gene prediction method: Protein Homology.;old_locus_tag=MAP4350c,MAP_4350c;product=50S ribosomal protein L34;protein_id=WP_003874369.1;transl_table=11;translation=MAKGKRTFQPNNRRRARVHGFRLRMRTRAGRAIVSGRRRKGRRALSA'

        at org.snpeff.snpEffect.factory.SnpEffPredictorFactoryGff.create(SnpEffPredictorFactoryGff.java:353)
        at org.snpeff.snpEffect.commandLine.SnpEffCmdBuild.run(SnpEffCmdBuild.java:369)
        at org.snpeff.SnpEff.run(SnpEff.java:1183)
        at org.snpeff.SnpEff.main(SnpEff.java:162)
$ snippy --version ; snpEff -version
snippy 3.2-dev
SnpEff  4.3t    2017-11-24

Any ideas?

@tseemann

This comment has been minimized.

Owner

tseemann commented May 12, 2018

@stephenturner snpEff changes its behaviour yet again, or fixed a bug...

what do you have in the main snippy script at this section?

    # it seems to be writeing phase=1 (aka frame) instead of 0 (0-based)
    # i suspect it is using /codon_start= incorrectly (1-based) !!!
    $f->frame(0);

The frame(0) line might be commented out in yours, which was to fix that bug in an older snpEff...

Change it and it should work.

I'm trying to make a release of 4.0 ASAP as this is causing lots of problems.

@stephenturner

This comment has been minimized.

stephenturner commented May 14, 2018

No, the $f->frame(0); is still in play. Here's what that whole block looks like.

while (my $seq = $in->next_seq) {
  exists $refseq{$seq->id} and err("Duplicate sequence ".$seq->id." in $reference");
  $refseq{ $seq->id } = uc($seq->seq); # keep for masking later
  $out->write_seq($seq);
  $nseq++;
  for my $f ($seq->get_SeqFeatures) {
    next if $f->primary_tag =~ m/^(source|misc_feature|gene|RBS)$/;
    $f->source_tag($EXE);
    # it seems to be writeing phase=1 (aka frame) instead of 0 (0-based)
    # i suspect it is using /codon_start= incorrectly (1-based) !!!
    $f->frame(0);
    if ($f->has_tag('locus_tag')) {
      my($id) = $f->get_tag_values('locus_tag');
      $f->add_tag_value('ID', $id);
    }
    if ($f->has_tag('gene')) {
      my($gene) = $f->get_tag_values('gene');
      $f->add_tag_value('Name', $gene);
    }
    $gff->write_feature($f);
    $nfeat++;
  }
}
$ snippy --version
snippy 3.2-dev

(downgraded to 3.2 at @Slugger70's suggestion #154 (comment))

@tseemann

This comment has been minimized.

Owner

tseemann commented May 18, 2018

I am planning to replace snpEff with bcftools csq ASAP.

@tseemann tseemann closed this Jul 7, 2018

@diaz13

This comment has been minimized.

diaz13 commented Aug 6, 2018

hi,
I installed snippy today.
And i run this command :
snippy --outdir --ref ../../../Parsnp-Linux64-v1.2/FTNF002-00.GBK --ctgs ../../scapper-master/medusa/*.fasta
[11:28:37] This is snippy 4.0-pre_20180729
[11:28:37] Written by Torsten Seemann
[11:28:37] Obtained from https://github.com/tseemann/snippy
[11:28:37] Detected operating system: linux
[11:28:37] Enabling bundled linux tools.
[11:28:37] Found bwa - /home/user/Tools/snippy-master/snippy/binaries/linux/bwa
[11:28:37] Found minimap2 - /home/user/Tools/minimap2-master//minimap2
[11:28:37] Found bcftools - /home/user/Tools/snippy-master/snippy/binaries/linux/bcftools
[11:28:37] Found samtools - /home/user/Tools/snippy-master/snippy/binaries/linux/samtools
[11:28:37] Found java - /usr/bin/java
[11:28:37] Found snpEff - /home/user/Tools/snippy-master/snippy/binaries/noarch/snpEff
[11:28:37] Found samclip - /home/user/Tools/snippy-master/snippy/binaries/noarch/samclip
[11:28:37] Found seqtk - /home/user/Tools/seqtk-master/seqtk
[11:28:37] Found snp-sites - /usr/bin/snp-sites
[11:28:37] Found parallel - /home/user/Tools/snippy-master/snippy/binaries/noarch/parallel
[11:28:37] Found freebayes - /home/user/Tools/snippy-master/snippy/binaries/linux/freebayes
[11:28:37] Found freebayes-parallel - /home/user/Tools/snippy-master/snippy/binaries/noarch/freebayes-parallel
[11:28:37] Found fasta_generate_regions.py - /home/user/Tools/snippy-master/snippy/binaries/noarch/fasta_generate_regions.py
[11:28:37] Found vcfstreamsort - /home/user/Tools/snippy-master/snippy/binaries/linux/vcfstreamsort
[11:28:37] Found vcfuniq - /home/user/Tools/snippy-master/snippy/binaries/linux/vcfuniq
[11:28:37] Found vcffirstheader - /home/user/Tools/snippy-master/snippy/binaries/noarch/vcffirstheader
[11:28:37] Found gzip - /bin/gzip
[11:28:37] Found seqret - /usr/bin/seqret
[11:28:37] Found vt - /home/user/Tools/vt-master/vt/vt
[11:28:37] Found snippy-vcf_to_tab - /home/user/Tools/snippy-master/snippy/bin/snippy-vcf_to_tab
[11:28:37] Found snippy-vcf_report - /home/user/Tools/snippy-master/snippy/bin/snippy-vcf_report
[11:28:37] Checking version: samtools --version is >= 1.7 - ok, have 1.9
[11:28:37] Checking version: bcftools --version is >= 1.7 - ok, have 1.9
[11:28:37] Checking version: freebayes --version is >= 1.1 - ok, have 1.1
[11:28:37] Checking version: snpEff -version is >= 4.3 - ok, have 4.3
[11:28:37] Please supply a reference FASTA/GBK/EMBL file with --reference

I don't understand, why he don't found the reference ?
I have 👍

@diaz13

This comment has been minimized.

diaz13 commented Aug 6, 2018

hi,
I installed snippy today.
And i run this command :
snipp --outdir results --ref FTNF002-00.GBK --ctgs ../../scapper-master/medusa/*.fasta
[11:28:37] This is snippy 4.0-pre_20180729
[11:28:37] Written by Torsten Seemann
[11:28:37] Obtained from https://github.com/tseemann/snippy
[11:28:37] Detected operating system: linux
[11:28:37] Enabling bundled linux tools.
[11:28:37] Found bwa - /home/user/Tools/snippy-master/snippy/binaries/linux/bwa
[11:28:37] Found minimap2 - /home/user/Tools/minimap2-master//minimap2
[11:28:37] Found bcftools - /home/user/Tools/snippy-master/snippy/binaries/linux/bcftools
[11:28:37] Found samtools - /home/user/Tools/snippy-master/snippy/binaries/linux/samtools
[11:28:37] Found java - /usr/bin/java
[11:28:37] Found snpEff - /home/user/Tools/snippy-master/snippy/binaries/noarch/snpEff
[11:28:37] Found samclip - /home/user/Tools/snippy-master/snippy/binaries/noarch/samclip
[11:28:37] Found seqtk - /home/user/Tools/seqtk-master/seqtk
[11:28:37] Found snp-sites - /usr/bin/snp-sites
[11:28:37] Found parallel - /home/user/Tools/snippy-master/snippy/binaries/noarch/parallel
[11:28:37] Found freebayes - /home/user/Tools/snippy-master/snippy/binaries/linux/freebayes
[11:28:37] Found freebayes-parallel - /home/user/Tools/snippy-master/snippy/binaries/noarch/freebayes-parallel
[11:28:37] Found fasta_generate_regions.py - /home/user/Tools/snippy-master/snippy/binaries/noarch/fasta_generate_regions.py
[11:28:37] Found vcfstreamsort - /home/user/Tools/snippy-master/snippy/binaries/linux/vcfstreamsort
[11:28:37] Found vcfuniq - /home/user/Tools/snippy-master/snippy/binaries/linux/vcfuniq
[11:28:37] Found vcffirstheader - /home/user/Tools/snippy-master/snippy/binaries/noarch/vcffirstheader
[11:28:37] Found gzip - /bin/gzip
[11:28:37] Found seqret - /usr/bin/seqret
[11:28:37] Found vt - /home/user/Tools/vt-master/vt/vt
[11:28:37] Found snippy-vcf_to_tab - /home/user/Tools/snippy-master/snippy/bin/snippy-vcf_to_tab
[11:28:37] Found snippy-vcf_report - /home/user/Tools/snippy-master/snippy/bin/snippy-vcf_report
[11:28:37] Checking version: samtools --version is >= 1.7 - ok, have 1.9
[11:28:37] Checking version: bcftools --version is >= 1.7 - ok, have 1.9
[11:28:37] Checking version: freebayes --version is >= 1.1 - ok, have 1.1
[11:28:37] Checking version: snpEff -version is >= 4.3 - ok, have 4.3
[11:28:37] Please supply a reference FASTA/GBK/EMBL file with --reference

@tseemann

This comment has been minimized.

Owner

tseemann commented Aug 7, 2018

@diaz13 rename your file to use a lowercase .gbk instead of .GBK
Bioperl might not understand .GBK

@diaz13

This comment has been minimized.

diaz13 commented Aug 8, 2018

He works but sometimes he puts me the same error message. But I will see that.
Thank you very much @tseemann

@AlcaArctica

This comment has been minimized.

AlcaArctica commented Aug 28, 2018

I just encountered the same isssue. Same problem with either a .gkb file from NCBI or one obtained with RAST. I am using snpEff -version 4.3 and snippy --version snippy 3.2-dev. Thanks for any advice

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment