Tiberius commands

Tiberius commands

Deep learning-based ab-initio gene structure prediction analysis

Go to the directory for this activity.

cd /home/train/Annotation_workshop/Tiberius

Example Tiberius run on a region of Arabidopsis using eudicotyledons weights (~1 minute)

(
tiberius.py \
    --genome Inputs/Genome/Athaliana_447_TAIR10.Chr3-1065466-1464870.unmasked.fa \
    --model Inputs/Weights/eudicotyledons \
    --out tiberius_unmasked.gtf
)

Analyse the effect of the masking on genomes.

Generate multiple annotation files using unmasked, repeat masked and fully masked genomes

Run Tiberius using the eudicotyledons weights (<2 minutes)

(
mkdir -p Masking_analysis/Annotations
for f in Inputs/Genome/{*unmasked*,*repeat_masked*,*fully_masked*} ; do
    tag=$(basename $f | rev | cut -d "." -f2 | rev )
    tiberius.py \
        --genome ${f} \
        --model Inputs/Weights/eudicotyledons \
        --out Masking_analysis/Annotations/tiberius.${tag}.gtf
done
)

Correct the GTF positions to match the original chromosome positions.

(
for f in Masking_analysis/Annotations/tiberius*gtf ; do
    tag=$(basename $f .gtf)_corrected
    ./Scripts/offset_gff.sh \
        $f \
        1065465 \
        > Masking_analysis/Annotations/${tag}.gtf
done
)

Create properly formatted GFF files

(
for f in Masking_analysis/Annotations/tiberius*corrected.gtf ; do
    tag=$(basename $f .gtf)
    ./Scripts/parse_tiberius_GTF_to_GFF3.py \
        --add_gff3_directives \
        $f \
        > Masking_analysis/Annotations/${tag}.gff
done
)

Use Mikado to generate metrics for each annotation.

(
mkdir -p Masking_analysis/Mikado_stats
for f in Masking_analysis/Annotations/tiberius*corrected.gff ; do
    tag=$(basename $f)
    mikado util stats $f Masking_analysis/Mikado_stats/${tag}.stats
done
)

Summarise the statistics generated by Mikado.

(
for f in Masking_analysis/Mikado_stats/*.stats ; do
    tag=$(basename $f)
    parse_mikado_stats \
        Masking_analysis/Mikado_stats/${tag} \
        > Masking_analysis/Mikado_stats/${tag}.summary
done
)

Concatenate all the summary files.

(
Scripts/paste_mikado_summary_stats.sh \
    Masking_analysis/Mikado_stats/*stats.summary |
    tabulate -s "\t" \
    > Masking_analysis/Mikado_stats/all_gff.summary
)

Visualise the final output.

cat Masking_analysis/Mikado_stats/all_gff.summary

Use Mikado to compare the annotations to the reference annotation.

(
for f in Masking_analysis/Annotations/*corrected.gff ; do
    tag=$(basename $f .gff)
    mikado compare \
        -r Inputs/Annotation/Athaliana_447_Araport11.Chr3-1065466-1464870.gene_exons.gff3 \
        -p $f  \
        -o Masking_analysis/Mikado_Compare/${tag}
done
)

Summarise all comparisons.

(
mikado util collect_compare \
    -fmt tsv \
    -l "all" \
    -o Masking_analysis/Mikado_Compare/Athaliana_tiberius_models \
    Masking_analysis/Mikado_Compare/*.stats
)

Visualise the F1 metrics of all the comparisons.

(
cat Masking_analysis/Mikado_Compare/Athaliana_tiberius_models.f1.tsv \
    | sed -e 's:Masking_analysis/Mikado_Compare/tiberius.::g' -e 's:_corrected.stats::g' \
    | tabulate -s "\t"
)

Analyse the effect of different weights when predicting protein-coding genes

Run Tiberius against select models to predict protein-coding genes.

Weights/models trained with soft-masking (<2 minutes)

(
mkdir -p Model_analysis/Annotations
for f in Inputs/Weights/{eudicotyledons,insecta,mammalia} ; do
    tag=$(basename $f)
    tiberius.py \
        --genome Inputs/Genome/Athaliana_447_TAIR10.Chr3-1065466-1464870.unmasked.fa \
        --model ${f} \
        --out Model_analysis/Annotations/tiberius.${tag}.gtf
done
)

Weights/models not trained for soft-masking (~2 minutes)

Note the usage of --no_softmasking option

(
mkdir -p Model_analysis/Annotations
for f in Inputs/Weights/{insecta_nomask,mammalia_nomask,vertebrates_nomask,fungi_nomask} ; do
    tag=$(basename $f)
    tiberius.py \
        --genome Inputs/Genome/Athaliana_447_TAIR10.Chr3-1065466-1464870.unmasked.fa \
        --model ${f} \
        --no_softmasking \
        --out Model_analysis/Annotations/tiberius.${tag}.gtf
done
)

Correct the GTF positions to match the original chromosome positions.

(
for f in Model_analysis/Annotations/tiberius*gtf ; do
    tag=$(basename $f .gtf)_corrected
    ./Scripts/offset_gff.sh \
        $f \
        1065465 \
        > Model_analysis/Annotations/${tag}.gtf
done
)

Create properly formatted GFF files

(
for f in Model_analysis/Annotations/tiberius*corrected.gtf ; do
    tag=$(basename $f .gtf)
    ./Scripts/parse_tiberius_GTF_to_GFF3.py \
        --add_gff3_directives \
        $f \
        > Model_analysis/Annotations/${tag}.gff
done
)

Use Mikado to generate metrics for each annotation.

(
mkdir -p Model_analysis/Mikado_stats
for f in Model_analysis/Annotations/tiberius*corrected.gff ; do
    tag=$(basename $f)
    mikado util stats $f Model_analysis/Mikado_stats/${tag}.stats
done
)

Summarise the statistics generated by Mikado.

(
for f in Model_analysis/Mikado_stats/*.stats ; do
    tag=$(basename $f)
    parse_mikado_stats \
        Model_analysis/Mikado_stats/${tag} \
        > Model_analysis/Mikado_stats/${tag}.summary
done
)

Concatenate all the summary files into one.

(
Scripts/paste_mikado_summary_stats.sh \
    Model_analysis/Mikado_stats/*stats.summary |
    tabulate -s "\t" \
    > Model_analysis/Mikado_stats/all_gff.summary
)

Visualise the final output.

cat Model_analysis/Mikado_stats/all_gff.summary

Use Mikado to compare the annotations to the reference annotation.

(
for f in Model_analysis/Annotations/*corrected.gff ; do
    tag=$(basename $f .gff)
    mikado compare \
        -r Inputs/Annotation/Athaliana_447_Araport11.Chr3-1065466-1464870.gene_exons.gff3 \
        -p $f  \
        -o Model_analysis/Mikado_Compare/${tag}
done
)

Summarise all comparisons.

(
mikado util collect_compare \
    -fmt tsv \
    -l "all" \
    -o Model_analysis/Mikado_Compare/Athaliana_tiberius_models \
    Model_analysis/Mikado_Compare/*.stats
)

Visualise the F1 metrics of all the comparisons.

(
cat Model_analysis/Mikado_Compare/Athaliana_tiberius_models.f1.tsv \
    | sed -e 's:Model_analysis/Mikado_Compare/tiberius.::g' -e 's:_corrected.stats::g' \
    | tabulate -s "\t"
)

Commands to clean up, i.e. return to only the original files.

cd /home/train/Annotation_workshop/Tiberius
rm -rf -v !("Inputs"|"Scripts"|"commands.txt"|"Example_output")