Augustus Training (with optimisation)

General note:

Augustus comes with a fairly wide set of trained species, and training jobs can be submitted via the web service http://bioinf.uni-greifswald.de/webaugustus/training/create
Augustus can be trained manually using the supplied scripts http://bioinf.uni-greifswald.de/augustus/binaries/retraining.html
Maker pipeline provides a process for training Snap and Augustus
Braker allows for fully automated training of Augustus and GeneMark-EX from RNA-seq and protein homology data
BUSCO can be used to train augustus https://busco.ezlab.org/busco_userguide.html

1. Setup output directory:

cd /home/train/Annotation_workshop/Augustus/Training

rm -rf Output && mkdir -p Output

2. Convert GFF3 into Genbank format:

(
gff2gbSmallDNA.pl \
    Inputs/gold.gff \
    Inputs/Athaliana_447_TAIR10.Chr3.fa \
    1000 \
    Output/good_transcripts.full_models.gb \
    --overlap
)

⚠️ Windows users:

gff2gbSmallDNA.pl Inputs/gold.gff Inputs/Athaliana_447_TAIR10.Chr3.fa 1000 Output/good_transcripts.full_models.gb --overlap

3. Count number of models:

grep -c '^LOCUS' Output/good_transcripts.full_models.gb

If over (>) 1000 models:

3.1 Randomly split the above set into a training and a test set
randomSplit.pl Output/good_transcripts.full_models.gb 1000
3.2 Setup links
cd Output && ln -s good_transcripts.full_models.gb.test good_transcripts.gb && cd ..

else:

3.1 Setup links

cd Output && ln -s good_transcripts.full_models.gb good_transcripts.gb && cd ..

4. Randomly split the above set into a training and a test set (~200 models)

randomSplit.pl Output/good_transcripts.gb 200

5. Copy Augustus species config directory to current working directory to prevent accident modification of pre-trained Augustus species.

cp -a /opt/data/config Output/config

6. Create an Augustus metadata parameters files for your species.

new_species.pl --AUGUSTUS_CONFIG_PATH=$PWD/Output/config --species=ath_wksp_opt

7. Turn on the prediction of untranslated (UTR) regions (UTR `off` to `on`)

vim Output/config/species/ath_wksp_opt/ath_wksp_opt_parameters.cfg

8. Make an initial Augustus training. This creates parameter files for exon, intron and intergenic region in the directory `$AUGUSTUS_CONFIG_PATH/species/ath_wksp_opt`

(
etraining \
    Output/good_transcripts.gb.train \
    --AUGUSTUS_CONFIG_PATH=$PWD/Output/config \
    --species=ath_wksp_opt \
    --stopCodonExcludedFromCDS=false
)

⚠️ Windows users:

etraining Output/good_transcripts.gb.train --AUGUSTUS_CONFIG_PATH=$PWD/Output/config --species=ath_wksp_opt --stopCodonExcludedFromCDS=false

...
...
start codon frequencies: ATG(800)
# admissible start codons and their probabilities: ATG(1), CTG(0), TTG(0)
...
...

9. Check initial Augustus prediction accuracy (without optimisation) [< 2 minutes]

(
augustus \
    --AUGUSTUS_CONFIG_PATH=$PWD/Output/config \
    --species=ath_wksp_opt \
    Output/good_transcripts.gb.test \
    > Output/augustus.initial_test.txt
)

⚠️ Windows users:

augustus --AUGUSTUS_CONFIG_PATH=$PWD/Output/config --species=ath_wksp_opt Output/good_transcripts.gb.test > Output/augustus.initial_test.txt

10. Check the accuracy report

grep -A 22 Evaluation Output/augustus.initial_test.txt

...
...
*******      Evaluation of gene prediction     *******

---------------------------------------------\
                 | sensitivity | specificity |
---------------------------------------------|
nucleotide level |       0.978 |        0.81 |
---------------------------------------------/

----------------------------------------------------------------------------------------------------------\
           |  #pred |  #anno |      |    FP = false pos. |    FN = false neg. |             |             |
           | total/ | total/ |   TP |--------------------|--------------------| sensitivity | specificity |
           | unique | unique |      | part | ovlp | wrng | part | ovlp | wrng |             |             |
----------------------------------------------------------------------------------------------------------|
           |        |        |      |                388 |                165 |             |             |
exon level |   1493 |   1270 | 1105 | ------------------ | ------------------ |        0.87 |        0.74 |
           |   1493 |   1270 |      |  105 |    3 |  280 |  107 |    3 |   55 |             |             |
----------------------------------------------------------------------------------------------------------/

----------------------------------------------------------------------------\
transcript | #pred | #anno |   TP |   FP |   FN | sensitivity | specificity |
----------------------------------------------------------------------------|
gene level |   310 |   200 |  110 |  200 |   90 |        0.55 |       0.355 |
----------------------------------------------------------------------------/

What the above means is that:

Of the 200 genes, 110 were predicted exactly  
  87% of the exons were predicted exactly  
  74% of the predicted exons were exactly as in the test set.

11. Start the optimize_augustus.pl script to optimise the prediction accuracy by adjusting the meta parameters in the `*_parameters.cfg` file. This optimisation can take quite some time.

(
optimize_augustus.pl \
    --AUGUSTUS_CONFIG_PATH=$PWD/Output/config \
    --species=ath_wksp_opt \
    Output/good_transcripts.gb.train \
    --cpus=4 \
    --kfold=8 \
    --UTR=on
)

⚠️ Windows users:

optimize_augustus.pl --AUGUSTUS_CONFIG_PATH=$PWD/Output/config --species=ath_wksp_opt Output/good_transcripts.gb.train --cpus=4 --kfold=8 --UTR=on

12. After optimize_augustus.pl has finished or (after you have interrupted it) you should (re)train AUGUSTUS

(
etraining \
    Output/good_transcripts.gb.train \
    --AUGUSTUS_CONFIG_PATH=$PWD/Output/config \
    --species=ath_wksp_opt \
    --stopCodonExcludedFromCDS=false
)

⚠️ Windows users:

etraining Output/good_transcripts.gb.train --AUGUSTUS_CONFIG_PATH=$PWD/Output/config --species=ath_wksp_opt --stopCodonExcludedFromCDS=false

13. Check final Augustus prediction accuracy (post optimisation) [< 2 minutes]

(
augustus \
    --AUGUSTUS_CONFIG_PATH=$PWD/Output/config \
    --species=ath_wksp_opt \
    Output/good_transcripts.gb.test \
    > Output/augustus.final_test.txt
)

⚠️ Windows users:

augustus --AUGUSTUS_CONFIG_PATH=$PWD/Output/config --species=ath_wksp_opt Output/good_transcripts.gb.test > Output/augustus.final_test.txt

14. Check the accuracy report, post-optimisation

grep -A 22 Evaluation Output/augustus.final_test.txt

if we run optimise Augustus with 4 CPUs and 8 kfold [~6 hrs]

*******      Evaluation of gene prediction     *******

---------------------------------------------\
                 | sensitivity | specificity |
---------------------------------------------|
nucleotide level |        0.98 |       0.801 |
---------------------------------------------/

----------------------------------------------------------------------------------------------------------\
           |  #pred |  #anno |      |    FP = false pos. |    FN = false neg. |             |             |
           | total/ | total/ |   TP |--------------------|--------------------| sensitivity | specificity |
           | unique | unique |      | part | ovlp | wrng | part | ovlp | wrng |             |             |
----------------------------------------------------------------------------------------------------------|
           |        |        |      |                406 |                149 |             |             |
exon level |   1527 |   1270 | 1121 | ------------------ | ------------------ |       0.883 |       0.734 |
           |   1527 |   1270 |      |   98 |    3 |  305 |  101 |    3 |   45 |             |             |
----------------------------------------------------------------------------------------------------------/

----------------------------------------------------------------------------\
transcript | #pred | #anno |   TP |   FP |   FN | sensitivity | specificity |
----------------------------------------------------------------------------|
gene level |   316 |   200 |  109 |  207 |   91 |       0.545 |       0.345 |
----------------------------------------------------------------------------/

What the above means is that:

Of the 200 genes, 109 were predicted exactly (9 genes fewer compared to 32 CPUs and 32 kfold run below)
  88.3% of the exons were predicted exactly  
  73.4% of the predicted exons were exactly as in the test set.

if we run optimise Augustus with 32 CPUs and 32 kfold [~40 minutes]

*******      Evaluation of gene prediction     *******

---------------------------------------------\
                 | sensitivity | specificity |
---------------------------------------------|
nucleotide level |       0.983 |       0.807 |
---------------------------------------------/

----------------------------------------------------------------------------------------------------------\
           |  #pred |  #anno |      |    FP = false pos. |    FN = false neg. |             |             |
           | total/ | total/ |   TP |--------------------|--------------------| sensitivity | specificity |
           | unique | unique |      | part | ovlp | wrng | part | ovlp | wrng |             |             |
----------------------------------------------------------------------------------------------------------|
           |        |        |      |                377 |                131 |             |             |
exon level |   1516 |   1270 | 1139 | ------------------ | ------------------ |       0.897 |       0.751 |
           |   1516 |   1270 |      |   82 |    2 |  293 |   85 |    2 |   44 |             |             |
----------------------------------------------------------------------------------------------------------/

----------------------------------------------------------------------------\
transcript | #pred | #anno |   TP |   FP |   FN | sensitivity | specificity |
----------------------------------------------------------------------------|
gene level |   312 |   200 |  118 |  194 |   82 |        0.59 |       0.378 |
----------------------------------------------------------------------------/

What the above means is that:

Of the 200 genes, 118 were predicted exactly  (9 genes more compared to 4 CPUs and 8 kfold run above)
  89.7% of the exons were predicted exactly  
  75.1% of the predicted exons were exactly as in the test set.

15. Backup the final Augustus config directory

cp -a Output/config Output/ath_wksp_opt_config

Uh oh!

Augustus Training (with optimisation)

Augustus Training (with optimisation)

1. Setup output directory:

2. Convert GFF3 into Genbank format:

3. Count number of models:

3.1 Randomly split the above set into a training and a test set

3.2 Setup links

3.1 Setup links

4. Randomly split the above set into a training and a test set (~200 models)

5. Copy Augustus species config directory to current working directory to prevent accident modification of pre-trained Augustus species.

6. Create an Augustus metadata parameters files for your species.

7. Turn on the prediction of untranslated (UTR) regions (UTR off to on)

8. Make an initial Augustus training. This creates parameter files for exon, intron and intergenic region in the directory $AUGUSTUS_CONFIG_PATH/species/ath_wksp_opt

9. Check initial Augustus prediction accuracy (without optimisation) [< 2 minutes]

10. Check the accuracy report

11. Start the optimize_augustus.pl script to optimise the prediction accuracy by adjusting the meta parameters in the *_parameters.cfg file. This optimisation can take quite some time.

12. After optimize_augustus.pl has finished or (after you have interrupted it) you should (re)train AUGUSTUS

13. Check final Augustus prediction accuracy (post optimisation) [< 2 minutes]

14. Check the accuracy report, post-optimisation

if we run optimise Augustus with 4 CPUs and 8 kfold [~6 hrs]

if we run optimise Augustus with 32 CPUs and 32 kfold [~40 minutes]

15. Backup the final Augustus config directory

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Clone this wiki locally

7. Turn on the prediction of untranslated (UTR) regions (UTR `off` to `on`)

8. Make an initial Augustus training. This creates parameter files for exon, intron and intergenic region in the directory `$AUGUSTUS_CONFIG_PATH/species/ath_wksp_opt`

11. Start the optimize_augustus.pl script to optimise the prediction accuracy by adjusting the meta parameters in the `*_parameters.cfg` file. This optimisation can take quite some time.