Skip to content

Helixer commands

Jump to bottom
Gemy George Kaithakottil edited this page Apr 22, 2026 · 4 revisions

Helixer commands

First steps.

Go to the directory for this activity.

cd /home/train/Annotation_workshop/Helixer

Example run.

(
Helixer.py \
    --fasta-path Inputs/Genome/Athaliana_447_TAIR10.Chr3-1065466-1464870.fa \
    --gff-output-path test.gff \
    --species Arabidopsis \
    --temporary-dir /tmp \
    --model-filepath Inputs/Models/land_plant_v0.3_a_0080.h5 \
    --batch-size 32 \
    --overlap-offset 53460 \
    --overlap-core-length 80190  \
    --subsequence-length 106920  \
    --min-coding-length 60
)

⚠️ Windows users:

Helixer.py --fasta-path Inputs/Genome/Athaliana_447_TAIR10.Chr3-1065466-1464870.fa --gff-output-path test.gff --species Arabidopsis --temporary-dir /tmp --model-filepath Inputs/Models/land_plant_v0.3_a_0080.h5 --batch-size 32 --overlap-offset 53460 --overlap-core-length 80190 --subsequence-length 106920 --min-coding-length 60

Pre-Processing.

Reformat the fasta file with the genome to an h5 format to be used by Helixer.

(
mkdir -p Format_conversion
fasta2h5.py \
    --species A_thaliana \
    --h5-output-path Format_conversion/A_thaliana.h5 \
    --fasta-path Inputs/Genome/Athaliana_447_TAIR10.Chr3-1065466-1464870.fa \
    --subsequence-length 106920
)

⚠️ Windows users:

mkdir -p Format_conversion; fasta2h5.py --species A_thaliana --h5-output-path Format_conversion/A_thaliana.h5 --fasta-path Inputs/Genome/Athaliana_447_TAIR10.Chr3-1065466-1464870.fa --subsequence-length 106920

Analysis of model effect in the output.

Use all the models available to predict protein-coding genes.

(
mkdir -p Predictions
for f in Inputs/Models/* ; do
    tag=$(basename $f | rev | cut -f2- -d "." | rev)
    HybridModel.py \
        --load-model-path $f \
        --test-data Format_conversion/A_thaliana.h5  \
        --overlap-offset 53460 \
        --core-length 106920  \
        --cpus 2 \
        --overlap \
        --val-test-batch-size 32 \
        -v \
        -p Predictions/${tag}_predictions.h5
done
)

⚠️ Windows users:

mkdir -p Predictions; for f in Inputs/Models/* ; do tag=$(basename $f | rev | cut -f2- -d "." | rev); HybridModel.py --load-model-path $f --test-data Format_conversion/A_thaliana.h5 --overlap-offset 53460 --core-length 106920 --cpus 2 --overlap --val-test-batch-size 32 -v -p Predictions/${tag}_predictions.h5; done

Transform the predictions into a GFF format file annotation.

(
mkdir -p Model_analysis/Annotations
for f in Predictions/* ; do
    tag=$(basename $f | rev | cut -f2- -d "." | rev)
    helixer_post_bin \
        Format_conversion/A_thaliana.h5 \
        ${f} \
        100 \
        0.1 \
        0.8 \
        60 \
        Model_analysis/Annotations/A_thaliana.helixer_${tag}.gff
done
)

⚠️ Windows users:

mkdir -p Model_analysis/Annotations; for f in Predictions/* ; do tag=$(basename $f | rev | cut -f2- -d "." | rev); helixer_post_bin Format_conversion/A_thaliana.h5 ${f} 100 0.1 0.8 60 Model_analysis/Annotations/A_thaliana.helixer_${tag}.gff; done

Use Mikado to generate metrics for each annotation.

(
mkdir -p Model_analysis/Mikado_stats
for f in Model_analysis/Annotations/* ; do
    tag=$(basename $f | rev | cut -f2- -d "." | rev)
    mikado util stats $f Model_analysis/Mikado_stats/${tag}.stats
done
)

⚠️ Windows users:

mkdir -p Model_analysis/Mikado_stats; for f in Model_analysis/Annotations/* ; do tag=$(basename $f | rev | cut -f2- -d "." | rev); mikado util stats $f Model_analysis/Mikado_stats/${tag}.stats; done

Summarise the statistics generated by Mikado.

(
for f in Model_analysis/Annotations/* ; do
    tag=$(basename $f | rev | cut -f2- -d "." | rev)
    parse_mikado_stats \
        Model_analysis/Mikado_stats/${tag}.stats \
        > Model_analysis/Mikado_stats/${tag}.stats.summary
done
)

⚠️ Windows users:

for f in Model_analysis/Annotations/* ; do tag=$(basename $f | rev | cut -f2- -d "." | rev); parse_mikado_stats Model_analysis/Mikado_stats/${tag}.stats > Model_analysis/Mikado_stats/${tag}.stats.summary; done

Concatenate all the summary files into one.

(
Scripts/paste_mikado_summary_stats.sh \
    Model_analysis/Mikado_stats/*stats.summary  \
    > Model_analysis/Mikado_stats/all_gff.summary
)

⚠️ Windows users:

Scripts/paste_mikado_summary_stats.sh Model_analysis/Mikado_stats/*stats.summary > Model_analysis/Mikado_stats/all_gff.summary

Visualise the final output.

cat Model_analysis/Mikado_stats/all_gff.summary
	Model_analysis/Mikado_stats/A_thaliana.helixer_fungi_v0.3_a_0100_predictions.stats.summary  Model_analysis/Mikado_stats/A_thaliana.helixer_invertebrate_v0.3_m_0100_predictions.stats.summary   Model_analysis/Mikado_stats/A_thaliana.helixer_land_plant_v0.3_a_0080_predictions.stats.summary Model_analysis/Mikado_stats/A_thaliana.helixer_vertebrate_v0.3_m_0080_predictions.stats.summary
Number of genes                   150.00   141.00   136.00   81.00
Number of Transcripts             150.00   141.00   136.00   81.00
Transcripts per gene              1.00     1.00     1.00     1.00
Number of monoexonic genes        35.00    20.00    25.00    7.00
Monoexonic transcripts            35.00    20.00    25.00    7.00
Transcript mean size cDNA (bp)    1225.80  1305.14  1318.44  1280.90
Transcript median size cDNA (bp)  1107.00  1121.00  1112.00  989.00
Min cDNA                          75.00    80.00    62.00    83.00
Max cDNA                          4435.00  5817.00  5537.00  5486.00
Total exons                       749.00   784.00   717.00   595.00
Exons per transcript              4.99     5.56     5.27     7.35
Exon mean size (bp)               245.49   234.73   250.08   174.37
CDS mean size (bp)                192.64   193.96   200.10   142.56
Transcript mean size CDS (bp)     922.10   978.04   1009.35  980.31
Transcript median size CDS (bp)   702.00   735.00   800.00   699.00
Min CDS                           69.00    78.00    60.00    63.00
Max CDS                           3867.00  5112.00  5112.00  5112.00
Intron mean size (bp)             115.36   163.15   151.25   237.60
5'UTR mean size (bp)              130.35   115.96   118.04   120.37
3'UTR mean size (bp)              173.35   211.13   191.05   180.22

Correct the gff positions to match the original chromosome positions.

(
for f in Model_analysis/Annotations/A_thaliana.helixer_* ; do
    tag=$(basename $f | rev | cut -f2- -d "." | rev)
    ./Scripts/offset_gff.sh \
        $f \
        1065465 \
        > Model_analysis/Annotations/${tag}_corrected.gff
done
)

⚠️ Windows users:

for f in Model_analysis/Annotations/A_thaliana.helixer_* ; do tag=$(basename $f | rev | cut -f2- -d "." | rev); ./Scripts/offset_gff.sh $f 1065465 > Model_analysis/Annotations/${tag}_corrected.gff; done

Use Mikado to compare the annotations to the reference annotation.

(
for f in Model_analysis/Annotations/*corrected* ; do
    tag=$(basename $f | rev | cut -f2- -d "." | rev)
    mikado compare \
        -r Inputs/Annotation/Athaliana_447_Araport11.Chr3-1065466-1464870.gene_exons.gff3 \
        -p $f  \
        -o Model_analysis/Mikado_Compare/${tag}
done
)

⚠️ Windows users:

for f in Model_analysis/Annotations/*corrected* ; do tag=$(basename $f | rev | cut -f2- -d "." | rev); mikado compare -r Inputs/Annotation/Athaliana_447_Araport11.Chr3-1065466-1464870.gene_exons.gff3 -p $f -o Model_analysis/Mikado_Compare/${tag}; done

Summarise all comparisons.

(
mikado util collect_compare \
    -fmt tsv \
    -l "all" \
    -o Model_analysis/Mikado_Compare/Athaliana_helixer_models \
    Model_analysis/Mikado_Compare/A_thaliana.helixer_*.stats
)

⚠️ Windows users:

mikado util collect_compare -fmt tsv -l "all" -o Model_analysis/Mikado_Compare/Athaliana_helixer_models Model_analysis/Mikado_Compare/A_thaliana.helixer_*.stats

Visualise the F1 metrics of all the comparisons.

less -S Model_analysis/Mikado_Compare/Athaliana_helixer_models.f1.tsv

Analyse the effect of the post-bin parameter.

Generate multiple annotation files using a different peak-threshold parameter.

(
mkdir -p Land_plant_analysis/Annotations
for f in {0.8,0.85,0.9,0.95,0.975} ; do
    helixer_post_bin \
        Format_conversion/A_thaliana.h5 \
        Predictions/land_plant_v0.3_a_0080_predictions.h5 \
        100 \
        0.1 \
        ${f} \
        60 \
        Land_plant_analysis/Annotations/A_thaliana.helixer_${f}_land_plant_v0.3_a_0080.gff
done
)

⚠️ Windows users:

mkdir -p Land_plant_analysis/Annotations; for f in {0.8,0.85,0.9,0.95,0.975} ; do helixer_post_bin Format_conversion/A_thaliana.h5 Predictions/land_plant_v0.3_a_0080_predictions.h5 100 0.1 ${f} 60 Land_plant_analysis/Annotations/A_thaliana.helixer_${f}_land_plant_v0.3_a_0080.gff; done

Use Mikado to generate metrics for each annotation.

(
mkdir -p Land_plant_analysis/Mikado_stats
for f in Land_plant_analysis/Annotations/* ; do
    tag=$(basename $f | rev | cut -f2- -d "." | rev)
    mikado util stats $f Land_plant_analysis/Mikado_stats/${tag}.stats
done
)

⚠️ Windows users:

mkdir -p Land_plant_analysis/Mikado_stats; for f in Land_plant_analysis/Annotations/* ; do tag=$(basename $f | rev | cut -f2- -d "." | rev); mikado util stats $f Land_plant_analysis/Mikado_stats/${tag}.stats; done

Summarise the statistics generated by Mikado.

(
for f in Land_plant_analysis/Annotations/* ; do
    tag=$(basename $f | rev | cut -f2- -d "." | rev)
    parse_mikado_stats \
        Land_plant_analysis/Mikado_stats/${tag}.stats \
        > Land_plant_analysis/Mikado_stats/${tag}.stats.summary
done
)

⚠️ Windows users:

for f in Land_plant_analysis/Annotations/* ; do tag=$(basename $f | rev | cut -f2- -d "." | rev); parse_mikado_stats Land_plant_analysis/Mikado_stats/${tag}.stats > Land_plant_analysis/Mikado_stats/${tag}.stats.summary; done

Concatenate all the summary files into one.

(
Scripts/paste_mikado_summary_stats.sh \
    Land_plant_analysis/Mikado_stats/*stats.summary \
    > Land_plant_analysis/Mikado_stats/all_gff.summary
)

⚠️ Windows users:

Scripts/paste_mikado_summary_stats.sh Land_plant_analysis/Mikado_stats/*stats.summary > Land_plant_analysis/Mikado_stats/all_gff.summary

Visualise the final output.

cat Land_plant_analysis/Mikado_stats/all_gff.summary
	Land_plant_analysis/Mikado_stats/A_thaliana.helixer_0.85_land_plant_v0.3_a_0080.stats.summary   Land_plant_analysis/Mikado_stats/A_thaliana.helixer_0.8_land_plant_v0.3_a_0080.stats.summary    Land_plant_analysis/Mikado_stats/A_thaliana.helixer_0.95_land_plant_v0.3_a_0080.stats.summary   Land_plant_analysis/Mikado_stats/A_thaliana.helixer_0.975_land_plant_v0.3_a_0080.stats.summary  Land_plant_analysis/Mikado_stats/A_thaliana.helixer_0.9_land_plant_v0.3_a_0080.stats.summary
Number of genes                   135.00   136.00   129.00   125.00   134.00
Number of Transcripts             135.00   136.00   129.00   125.00   134.00
Transcripts per gene              1.00     1.00     1.00     1.00     1.00
Number of monoexonic genes        25.00    25.00    22.00    20.00    25.00
Monoexonic transcripts            25.00    25.00    22.00    20.00    25.00
Transcript mean size cDNA (bp)    1325.05  1318.44  1371.62  1394.98  1329.69
Transcript median size cDNA (bp)  1116.00  1112.00  1176.00  1214.00  1117.00
Min cDNA                          62.00    62.00    74.00    74.00    62.00
Max cDNA                          5537.00  5537.00  5537.00  5537.00  5537.00
Total exons                       715.00   717.00   702.00   696.00   713.00
Exons per transcript              5.30     5.27     5.44     5.57     5.32
Exon mean size (bp)               250.18   250.08   252.05   250.53   249.90
CDS mean size (bp)                200.26   200.10   202.23   201.58   200.41
Transcript mean size CDS (bp)     1014.64  1009.35  1051.88  1072.42  1020.00
Transcript median size CDS (bp)   810.00   800.00   843.00   864.00   810.00
Min CDS                           60.00    60.00    66.00    66.00    60.00
Max CDS                           5112.00  5112.00  5112.00  5112.00  5112.00
Intron mean size (bp)             151.19   151.25   151.81   151.69   151.07
5'UTR mean size (bp)              118.71   118.04   122.84   125.24   119.43
3'UTR mean size (bp)              191.70   191.05   196.90   197.32   190.27

Correct the gff positions to match the original chromosome positions.

(
for f in Land_plant_analysis/Annotations/A_thaliana.helixer_* ; do
    tag=$(basename $f | rev | cut -f2- -d "." | rev)
    ./Scripts/offset_gff.sh \
        $f \
        1065465 \
        > Land_plant_analysis/Annotations/${tag}_corrected.gff
done
)

⚠️ Windows users:

for f in Land_plant_analysis/Annotations/A_thaliana.helixer_* ; do tag=$(basename $f | rev | cut -f2- -d "." | rev); ./Scripts/offset_gff.sh $f 1065465 > Land_plant_analysis/Annotations/${tag}_corrected.gff; done

Use Mikado to compare the annotations to the reference annotation.

(
for f in Land_plant_analysis/Annotations/*corrected* ; do
    tag=$(basename $f | rev | cut -f2- -d "." | rev)
    mikado compare \
        -r Inputs/Annotation/Athaliana_447_Araport11.Chr3-1065466-1464870.gene_exons.gff3 \
        -p $f  \
        -o Land_plant_analysis/Mikado_Compare/${tag}
done
)

⚠️ Windows users:

for f in Land_plant_analysis/Annotations/*corrected* ; do tag=$(basename $f | rev | cut -f2- -d "." | rev); mikado compare -r Inputs/Annotation/Athaliana_447_Araport11.Chr3-1065466-1464870.gene_exons.gff3 -p $f -o Land_plant_analysis/Mikado_Compare/${tag}; done

Summarise all comparisons.

(
mikado util collect_compare \
    -fmt tsv \
    -l "all" \
    -o Land_plant_analysis/Mikado_Compare/Athaliana_helixer_models \
    Land_plant_analysis/Mikado_Compare/A_thaliana.helixer_*.stats
)

⚠️ Windows users:

mikado util collect_compare -fmt tsv -l "all" -o Land_plant_analysis/Mikado_Compare/Athaliana_helixer_models Land_plant_analysis/Mikado_Compare/A_thaliana.helixer_*.stats

Visualise the F1 metrics of all the comparisons.

less -S Land_plant_analysis/Mikado_Compare/Athaliana_helixer_models.f1.tsv

Commands to clean up, i.e. return to only the original files.

cd /home/train/Annotation_workshop/Helixer
rm -rf -v !("Inputs"|"Scripts"|"commands.txt"|"Example_output")

⚠️ Windows users:

cd /home/train/Annotation_workshop/Helixer && rm -rf -v !("Inputs"|"Scripts"|"commands.txt"|"Example_output")

Clone this wiki locally