SNAP is a general purpose gene finding program suitable for both eukaryotic and prokaryotic genomes. SNAP is an acroynm for Semi-HMM-based Nucleic Acid Parser.
Korf I. Gene finding in novel Genomes. BMC Bioinformatics 2004, 5:59
I appreciate bug reports, comments, and suggestions. My current contact information is:
- email: ifkorf@ucdavis.edu
- web: http://korflab.ucdavis.edu
This software is covered by the MIT License.
DNA Contains some sample sequences
HMM Contains SNAP parameter files
LICENSE MIT license
Makefile For compiling
Makefile.include Automatically generated, should not be edited
Zoe A library containing lots of the base functions
fathom.c Utility for investigating sequences and annotation
forge.c Parameter estimation
hmm-assembler.pl Creates HMMs for SNAP
snap.c Gene prediction program
If you wish to train SNAP for a new genome, please contact me. Parameter estimation is not particularly difficult, but the procedure is not well documented and I have only included the minimum applications here. I've included the basic strategy at the end of this document.
The software is routinely compiled and tested on Mac and Linux on a variety of architectures. It should compile cleanly on any Linux/Unix type operating systems but new compilers sometimes complain, so please let me know if you have problems.
The ZOE environment variable can be used by SNAP to find the HMM files. Set this to the directory containing this file. For example, if you unpackaged the tar-ball in /usr/local, set the ZOE environment variable to /usr/local/Zoe
setenv ZOE /usr/local/Zoe # csh, tcsh, etc
export ZOE=/usr/local/Zoe # sh, bash, etc
If you do not use the ZOE environment variable, you can still use SNAP but you must specify the explict path to the parameter file.
Provided you have gcc and make, compiling should be as simple as:
make
Testing
./snap HMM/thale DNA/thale.dna.gz
./snap HMM/worm DNA/worm.dna.gz
Sequences must be in FASTA format. It's a good idea if you don't have genes that are too related to each other.
Gene structures must be in ZFF format. What is ZFF? It is a non-standard format (ie. nobody uses it but me) that bears resemblence to FASTA and GFF (both true standards). There are two styles of ZFF, the short format and the long format. In both cases, the sequence records are separated by a definition line, just like FASTA. In the short format, there are 4 fields: Label, Begin, End, Group. The 4th field is optional. Label is a controlled vocabulary (see zoeFeature.h for a complete list). All exons of a gene (or more appropriately a transcriptional unit) must share the same unique group name. The strand of the feature is implied in the coordinates, so if Begin > End, the feature is on the minus strand. Here's and example of the short format with two sequences, each containing a single gene on the plus strand:
>sequence-1
Einit 201 325 Y73E7A.6
Eterm 2175 2319 Y73E7A.6
>sequence-2
Einit 201 462 Y73E7A.7
Exon 1803 2031 Y73E7A.7
Exon 2929 3031 Y73E7A.7
Exon 3467 3624 Y73E7A.7
Exon 4185 4406 Y73E7A.7
Eterm 5103 5280 Y73E7A.7
The long format adds 5 fields between the coordinates and the group: Strand, Score, 5'-overhang, 3'-overhang, and Frame. Strand is +/-. Score is any floating point value. 5'- and 3'-overhang are the number of bp of an incomplete codon at each end of an exon. Frame is the reading frame (0..2 and not 1..3). Here's an example of the long format:
>Y73E7A.6
Einit 201 325 + 90 0 2 1 Y73E7A.6
Eterm 2175 2319 + 295 1 0 2 Y73E7A.6
>Y73E7A.7
Einit 201 462 + 263 0 1 1 Y73E7A.7
Exon 1803 2031 + 379 2 2 0 Y73E7A.7
Exon 2929 3031 + 236 1 0 0 Y73E7A.7
Exon 3467 3624 + 152 0 2 0 Y73E7A.7
Exon 4185 4406 + 225 1 2 2 Y73E7A.7
Eterm 5103 5280 + 46 1 0 2 Y73E7A.7
In this tutorial, we will create SNAP HMM files for 3 different genomes. In the
DATA directory, you will find fasta and gff3 files corresponding to 1 percent
of the A. thaliana, C. elegans, and D. melanogaster genomes. Let's start by
creating a directory for training A. thaliana in the main SNAP directory. We'll
run gff3_to_zff.pl to convert the annotation to ZFF.
mkdir train_at
cd train_at
../gff3_to_zff.pl ../DATA/at.fa.gz ../DATA/at.gff3.gz > at.zff
The next step is to check for errors in the annotation. The training procedure assumes that genes are canonical in various respects.
- Coding sequences start with ATG and end in a stop codon
- Splice sites follow the GT..AG rule
- Genes are at least 150 bp
- Exons are at least 6 bp
- CDS are at least 150 bp
- Introns are at least 30 bp
Running fathom -validate will tell you which genes look ok and which genes
look suspicious. Let's try one.
../fathom -validate at.zff ../DATA/at.fa.gz > at.validate
This will produce a bunch of output to STDERR. You will see several WARNING lines saying the DNA and annotation don't have the same definition lines. That's okay. The ZFF contains only the sequence id from the FASTA file and not the whole definition line present in the original FASTA. The last line gives some overall stats.
463 genes, 463 OK, 40 warnings, 0 errors
If you examine the at.validate file, you will see warnings for some short
genes, short exons, non-canonical introns, etc. We won't be using these to
train SNAP. To split genes into various categories use the -categorize
function of fathom and give it a value for how much intergenic sequence you
want on each side of a gene. For example fathom -categorize 1000 attempts to
put 1000 bp of genomic sequence on each side of a gene. However, if two genes
are close to each other, say only 400 bp apart, they split the intergenic
sequence and each get 200 bp of intergenic.
../fathom -categorize 100 at.zff ../DATA/at.fa.gz
This produces several new files:
alt.*contains genes with alternative splicingerr.*contains genes with errorsolp.*contains overlapping genesuni.*contains unique geneswrn.*contains genes with warnings
fathom doesn't want to create training files with alternative splicing. It
could create a case of overtraining for those specific genes. If you have a lot
of alternative splicing, you may want to remove all of the isoforms except for
the main one. fathom also doesn't know what to do with overlapping genes
because it requires genes to have intergenic sequence on either side. These
tend to be rare. Genes with unusual features or outright errors are separated
also.
The next step is to export all of the uni genes into their plus-stranded
versions.
../fathom -export 100 -plus uni.*
This creates 4 new files:
export.aacontains the amino acid sequencesexport.anncontains the annotation in ZFF formatexport.dnacontains the sequences in FASTA formatexport.txcontains the sequences of the coding sequences
Ideally, all of the proteins in export.aa start with M and end with *.
Similarly, the export.tx files should start with ATG and end in a stop
codon. All of the genes should validate without any reported warnings or
errors.
../fathom -validate export.ann export.dna
The next step is to run forge, which will create a large number of model
files.
../forge export.ann export.dna
Finally, run hmm-assembler.pl to glue the various models together to form an
hmm parameter file. There are several options, but we'll just use the defaults.
./hmm-assembler.pl A.thaliana . > at.hmm
To verify this works, you can try it on the various fasta files we've used.
../snap at.hmm export.dna
../snap at.hmm uni.dna
../snap at.hmm ../DATA/at.fa.gz
Next, we are going to try a slightly different training procedure that might be
better if you have a lot of genes that are getting stuffed into the wrn.*
files. Let's rewind a bit.
../fathom -categorize 100 at.zff ../DATA/at.fa.gz
Let's see how many genes are in each category.
grep -c ">" *.ann
alt.ann:77
err.ann:0
olp.ann:0
uni.ann:236
wrn.ann:33
The standard procedure will have us training from the 236 genes in the uni
category. To recover the genes in the wrn category, we'll just glue them to
the uni and then export that for the training.
cat uni.ann wrn.ann > glue.ann
cat uni.dna wrn.dna > glue.dna
../fathom -export 100 -plus glue.ann glue.dna
../forge export.ann export.dna
../hmm-assembler.pl whatever .
What about all of the genes in the alt category? Those genes are reported to
have multiple isoforms. Training from a gene with 10 isoforms would count that
gene 10 times, so these are generally skipped. However, as more and more
isoforms are found, this will become problematic. You will need some way to
figure out which isoform is canonical and delete the rest. I don't have an
automated way to do that as each community has their own standards.
Try the training procedures for the C. elegans and D. melanogaster genomes next. Note that these training sets represent just 1% of each chromosome and are just for demonstration purposes.