Augustus Training (with optimisation)

General note:

Augustus comes with a fairly wide set of trained species, and training jobs can be submitted via the web service http://bioinf.uni-greifswald.de/webaugustus/training/create
Augustus can be trained manually using the supplied scripts http://bioinf.uni-greifswald.de/augustus/binaries/retraining.html
Maker pipeline provides a process for training Snap and Augustus
Braker allows for fully automated training of Augustus and GeneMark-EX from RNA-seq and protein homology data
BUSCO can be used to train augustus https://busco.ezlab.org/busco_userguide.html

1. Setup output directory:

cd /home/train/Annotation_workshop/Augustus/Training

rm -rf Output && mkdir -p Output

2. Convert GFF3 into Genbank format:

gff2gbSmallDNA.pl Inputs/gold.gff Inputs/Athaliana_447_TAIR10.Chr3.fa 1000 Output/good_transcripts.full_models.gb --overlap

3. Count number of models:

grep -c '^LOCUS' Output/good_transcripts.full_models.gb

If over (>) 1000 models:

3.1 Randomly split the above set into a training and a test set
randomSplit.pl Output/good_transcripts.full_models.gb 1000
3.2 Setup links
cd Output && ln -s good_transcripts.full_models.gb.test good_transcripts.gb && cd ..

else:

3.1 Setup links

cd Output && ln -s good_transcripts.full_models.gb good_transcripts.gb && cd ..

4. Randomly split the above set into a training and a test set (~200 models)

randomSplit.pl Output/good_transcripts.gb 200

5. Copy Augustus species config directory to current working directory to prevent accident modification of pre-trained Augustus species.

cp -a /opt/data/config Output/config

6. Create an Augustus metadata parameters files for your species.

new_species.pl --AUGUSTUS_CONFIG_PATH=$PWD/Output/config --species=ath_wksp_opt

7. Turn on the prediction of untranslated (UTR) regions (UTR `off` to `on`)

vim.tiny Output/config/species/ath_wksp_opt/ath_wksp_opt_parameters.cfg

8. Make an initial Augustus training. This creates parameter files for exon, intron and intergenic region in the directory `$AUGUSTUS_CONFIG_PATH/species/ath_wksp_opt`

etraining Output/good_transcripts.gb.train --AUGUSTUS_CONFIG_PATH=$PWD/Output/config --species=ath_wksp_opt --stopCodonExcludedFromCDS=false

...
...
start codon frequencies: ATG(800)
# admissible start codons and their probabilities: ATG(1), CTG(0), TTG(0)
...
...

9. Check initial Augustus prediction accuracy (without optimisation) [< 2 minutes]

/usr/bin/time -v augustus --AUGUSTUS_CONFIG_PATH=$PWD/Output/config --species=ath_wksp_opt Output/good_transcripts.gb.test > Output/augustus.initial_test.txt

10. Check the accuracy report

grep -A 22 Evaluation Output/augustus.initial_test.txt

...
...
*******      Evaluation of gene prediction     *******

---------------------------------------------\
                 | sensitivity | specificity |
---------------------------------------------|
nucleotide level |       0.978 |       0.799 |
---------------------------------------------/

----------------------------------------------------------------------------------------------------------\
           |  #pred |  #anno |      |    FP = false pos. |    FN = false neg. |             |             |
           | total/ | total/ |   TP |--------------------|--------------------| sensitivity | specificity |
           | unique | unique |      | part | ovlp | wrng | part | ovlp | wrng |             |             |
----------------------------------------------------------------------------------------------------------|
           |        |        |      |                395 |                175 |             |             |
exon level |   1590 |   1370 | 1195 | ------------------ | ------------------ |       0.872 |       0.752 |
           |   1590 |   1370 |      |  115 |    6 |  274 |  118 |    6 |   51 |             |             |
----------------------------------------------------------------------------------------------------------/

----------------------------------------------------------------------------\
transcript | #pred | #anno |   TP |   FP |   FN | sensitivity | specificity |
----------------------------------------------------------------------------|
gene level |   307 |   200 |  104 |  203 |   96 |        0.52 |       0.339 |
----------------------------------------------------------------------------/

What the above means is that:

Of the 200 genes 104 were predicted exactly  
  87.2% of the exons were predicted exactly  
  75.2% of the predicted exons were exactly as in the test set.

11. Start the optimize_augustus.pl script to optimise the prediction accuracy by adjusting the meta parameters in the `*_parameters.cfg` file. This optimisation can take quite some time.

/usr/bin/time -v optimize_augustus.pl --AUGUSTUS_CONFIG_PATH=$PWD/Output/config --species=ath_wksp_opt Output/good_transcripts.gb.train --cpus=4 --kfold=8 --UTR=on

12. After optimize_augustus.pl has finished or (after you have interrupted it) you should (re)train AUGUSTUS

etraining Output/good_transcripts.gb.train --AUGUSTUS_CONFIG_PATH=$PWD/Output/config --species=ath_wksp_opt --stopCodonExcludedFromCDS=false

13. Check final Augustus prediction accuracy (post optimisation) [< 2 minutes]

augustus --AUGUSTUS_CONFIG_PATH=$PWD/Output/config --species=ath_wksp_opt Output/good_transcripts.gb.test > Output/augustus.final_test.txt

14. Check the accuracy report, post optimisation

grep -A 22 Evaluation Output/augustus.final_test.txt

if we run optimise Augustus with 4 CPUs and 8 kfold (~16 hrs)

*******      Evaluation of gene prediction     *******

---------------------------------------------\
                 | sensitivity | specificity |
---------------------------------------------|
nucleotide level |       0.986 |       0.796 |
---------------------------------------------/

----------------------------------------------------------------------------------------------------------\
           |  #pred |  #anno |      |    FP = false pos. |    FN = false neg. |             |             |
           | total/ | total/ |   TP |--------------------|--------------------| sensitivity | specificity |
           | unique | unique |      | part | ovlp | wrng | part | ovlp | wrng |             |             |
----------------------------------------------------------------------------------------------------------|
           |        |        |      |                399 |                138 |             |             |
exon level |   1631 |   1370 | 1232 | ------------------ | ------------------ |       0.899 |       0.755 |
           |   1631 |   1370 |      |  101 |    4 |  294 |  105 |    4 |   29 |             |             |
----------------------------------------------------------------------------------------------------------/

----------------------------------------------------------------------------\
transcript | #pred | #anno |   TP |   FP |   FN | sensitivity | specificity |
----------------------------------------------------------------------------|
gene level |   309 |   200 |  111 |  198 |   89 |       0.555 |       0.359 |
----------------------------------------------------------------------------/

What the above means is that:

Of the 200 genes 111 were predicted exactly  (an increase by 4 genes compared to 32 CPUs and 32 kfold run below)
  89.9% of the exons were predicted exactly  
  75.5% of the predicted exons were exactly as in the test set.

if we run optimise Augustus with 32 CPUs and 32 kfold (~1h 20m)

*******      Evaluation of gene prediction     *******

---------------------------------------------\
                 | sensitivity | specificity |
---------------------------------------------|
nucleotide level |       0.985 |       0.799 |
---------------------------------------------/

----------------------------------------------------------------------------------------------------------\
           |  #pred |  #anno |      |    FP = false pos. |    FN = false neg. |             |             |
           | total/ | total/ |   TP |--------------------|--------------------| sensitivity | specificity |
           | unique | unique |      | part | ovlp | wrng | part | ovlp | wrng |             |             |
----------------------------------------------------------------------------------------------------------|
           |        |        |      |                401 |                145 |             |             |
exon level |   1626 |   1370 | 1225 | ------------------ | ------------------ |       0.894 |       0.753 |
           |   1626 |   1370 |      |  107 |    5 |  289 |  110 |    5 |   30 |             |             |
----------------------------------------------------------------------------------------------------------/

----------------------------------------------------------------------------\
transcript | #pred | #anno |   TP |   FP |   FN | sensitivity | specificity |
----------------------------------------------------------------------------|
gene level |   303 |   200 |  108 |  195 |   92 |        0.54 |       0.356 |
----------------------------------------------------------------------------/

What the above means is that:

Of the 200 genes 108 were predicted exactly  (a decrease by 4 genes compared to 4 CPUs and 8 kfold run above)
  89.4% of the exons were predicted exactly  
  75.3% of the predicted exons were exactly as in the test set.

15. Backup the final Augustus config directory

cp -a Output/config Output/ath_wksp_opt_config

Uh oh!

Augustus Training (with optimisation)

Augustus Training (with optimisation)

1. Setup output directory:

2. Convert GFF3 into Genbank format:

3. Count number of models:

3.1 Randomly split the above set into a training and a test set

3.2 Setup links

3.1 Setup links

4. Randomly split the above set into a training and a test set (~200 models)

5. Copy Augustus species config directory to current working directory to prevent accident modification of pre-trained Augustus species.

6. Create an Augustus metadata parameters files for your species.

7. Turn on the prediction of untranslated (UTR) regions (UTR off to on)

8. Make an initial Augustus training. This creates parameter files for exon, intron and intergenic region in the directory $AUGUSTUS_CONFIG_PATH/species/ath_wksp_opt

9. Check initial Augustus prediction accuracy (without optimisation) [< 2 minutes]

10. Check the accuracy report

11. Start the optimize_augustus.pl script to optimise the prediction accuracy by adjusting the meta parameters in the *_parameters.cfg file. This optimisation can take quite some time.

12. After optimize_augustus.pl has finished or (after you have interrupted it) you should (re)train AUGUSTUS

13. Check final Augustus prediction accuracy (post optimisation) [< 2 minutes]

14. Check the accuracy report, post optimisation

if we run optimise Augustus with 4 CPUs and 8 kfold (~16 hrs)

if we run optimise Augustus with 32 CPUs and 32 kfold (~1h 20m)

15. Backup the final Augustus config directory

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Clone this wiki locally

7. Turn on the prediction of untranslated (UTR) regions (UTR `off` to `on`)

8. Make an initial Augustus training. This creates parameter files for exon, intron and intergenic region in the directory `$AUGUSTUS_CONFIG_PATH/species/ath_wksp_opt`

11. Start the optimize_augustus.pl script to optimise the prediction accuracy by adjusting the meta parameters in the `*_parameters.cfg` file. This optimisation can take quite some time.