Merge pull request #166 from bioinfo-chru-strasbourg/options_and_docs

Add annotation strategy (update, append) #165, add docs #4
bioinfo-chru-strasbourg · Mar 7, 2024 · 04aac00 · 04aac00
2 parents 84de157 + 95d52db
commit 04aac00
Show file tree

Hide file tree

Showing 11 changed files with 320 additions and 52 deletions.
diff --git a/docs/help.html b/docs/help.html
@@ -150,6 +150,16 @@ <H1>HOWARD Help</h1>
 Default assembly
 Default: &#x27;hg19&#x27;
 
+</pre><pre>--annotations_update
+Update option for annotation (Only for Parquet annotation).
+If True, annotation fields will be removed and re-annotated.
+These options will be applied to all annotation databases.default: False
+
+</pre><pre>--annotations_append
+Append option for annotation (Only for Parquet annotation).
+If True, annotation fields will be annotated only if not annotation exists for the variant.
+These options will be applied to all annotation databases.default: False
+
 </pre><H2>CALCULATION</H2>
 <p>Calculation processes variants information to generate new information, such as: identify variation type (VarType), harmonizes allele frequency (VAF) and calculate sttistics (VAF_stats), extracts Nomen (transcript, cNomen, pNomen...) from an HGVS field (e.g. snpEff, Annovar) with an optional list of personalized transcripts, generates VaRank format barcode, identify trio inheritance.</p>Usage examples:<br>&nbsp;&nbsp;&nbsp;howard calculation --input=tests/data/example.full.vcf --output=/tmp/example.calculation.tsv --calculations='vartype' <br>&nbsp;&nbsp;&nbsp;howard calculation --input=tests/data/example.ann.vcf.gz --output=/tmp/example.calculated.tsv --calculations='snpeff_hgvs,NOMEN' --hgvs_field=snpeff_hgvs --transcripts=tests/data/transcripts.tsv <br>&nbsp;&nbsp;&nbsp;howard calculation --show_calculations <br><H3>Main options</H3>
 <pre>--input=&lt;input&gt;

diff --git a/docs/help.md b/docs/help.md
@@ -347,6 +347,22 @@ Default assembly
 Default: 'hg19'
 ```
 
+```
+--annotations_update
+
+Update option for annotation (Only for Parquet annotation).
+If True, annotation fields will be removed and re-annotated.
+These options will be applied to all annotation databases.default: False
+```
+
+```
+--annotations_append
+
+Append option for annotation (Only for Parquet annotation).
+If True, annotation fields will be annotated only if not annotation exists for the variant.
+These options will be applied to all annotation databases.default: False
+```
+
 
 
 ## CALCULATION tool

diff --git a/docs/help.param.html b/docs/help.param.html
@@ -1,6 +1,6 @@
 <H1 id='HOWARD Parameters'>HOWARD Parameters</H1>HOWARD Parameters JSON file defined parameters to process annotations, prioritization, calculations, convertions and queries.<br>
                     <H2>Table of contents</H1>
-                    - <a href='#HOWARD Parameters'>HOWARD Parameters</a><BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;- <a href='#annotations'>annotations</a><BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;- <a href='#annotations::parquet'>parquet</a><BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;- <a href='#annotations::parquet::annotations'>annotations</a><BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;- <a href='#annotations::parquet::update'>update</a><BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;- <a href='#annotations::bcftools'>bcftools</a><BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;- <a href='#annotations::bcftools::annotations'>annotations</a><BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;- <a href='#annotations::annovar'>annovar</a><BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;- <a href='#annotations::annovar::annotations'>annotations</a><BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;- <a href='#annotations::annovar::options'>options</a><BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;- <a href='#annotations::snpeff'>snpeff</a><BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;- <a href='#annotations::snpeff::options'>options</a><BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;- <a href='#annotations::exomiser'>exomiser</a><BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;- <a href='#annotations::exomiser::release'>release</a><BR>
+                    - <a href='#HOWARD Parameters'>HOWARD Parameters</a><BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;- <a href='#annotations'>annotations</a><BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;- <a href='#annotations::parquet'>parquet</a><BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;- <a href='#annotations::parquet::annotations'>annotations</a><BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;- <a href='#annotations::bcftools'>bcftools</a><BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;- <a href='#annotations::bcftools::annotations'>annotations</a><BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;- <a href='#annotations::annovar'>annovar</a><BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;- <a href='#annotations::annovar::annotations'>annotations</a><BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;- <a href='#annotations::annovar::options'>options</a><BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;- <a href='#annotations::snpeff'>snpeff</a><BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;- <a href='#annotations::snpeff::options'>options</a><BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;- <a href='#annotations::exomiser'>exomiser</a><BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;- <a href='#annotations::exomiser::release'>release</a><BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;- <a href='#annotations::options'>options</a><BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;- <a href='#annotations::options::update'>update</a><BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;- <a href='#annotations::options::append'>append</a><BR>
                     <br>
                     <H2 id='annotations'>annotations</H2>Annotation process using HOWARD algorithms or external tools.<br>For HOWARD Parquet algorithm, specify the list of database files available (formats such as Parquet, VCF, TSV, duckDB, JSON). This parameter enables users to select specific database fields and optionally rename them. Use 'INFO' keyword to select all fields within the database. If a full path is not provided, the system will automatically detect files within database folders (see Configuration doc) and assembly (see Parameter option).<br>For external tools, such as Annovar, snpEff and Exomiser, specify parameters such as annotation keywords (Annovar) and options (depending on the tool).<br>Examples: <xmp># Annotation with multiple tools in multiple formats with multiple options
 "annotation": {
@@ -63,8 +63,7 @@ <H2 id='annotations'>annotations</H2>Annotation process using HOWARD algorithms
       "REVEL_rankscore": null
    }
 }
-</xmp><br><H4 id='annotations::parquet::update'>annotations::parquet::update</H4>Update option for Parquet annotation. If True, annotation fields will be updated if exists. If False, annotation fields will not change if it already exists. These options will be applied to all annotation databases.<br>Examples: <xmp># Apply update on all annotation fields for all databases.
-"update": True</xmp><br><H3 id='annotations::bcftools'>annotations::bcftools</H3>Annotation process using BCFTools. Provide a list of database files and annotation fields.<br>Examples: <xmp># Annotation with multiple databases in multiple formats
+</xmp><br><H3 id='annotations::bcftools'>annotations::bcftools</H3>Annotation process using BCFTools. Provide a list of database files and annotation fields.<br>Examples: <xmp># Annotation with multiple databases in multiple formats
 "parquet": {
    "bcftools": {
       "/path/to/database1.vcf.gz": {
@@ -146,12 +145,15 @@ <H2 id='annotations'>annotations</H2>Annotation process using HOWARD algorithms
       " -hgvs -noShiftHgvs -spliceSiteSize 3 -lof -oicr "}
    }
 }</xmp><br><H4 id='annotations::snpeff::options'>annotations::snpeff::options</H4>String (as command line) of options available such as:<br> - filters on variants (regions filter, specific changes as intronic or downstream)<br> - annotation (e.g. HGVS, loss of function) <br> - database (e.g. only protein coding transcripts, splice sites size)<br>Examples: <xmp># Annotation with snpEff databases, with options to generate HGVS annotation, specify to not shift variants according to HGVS notation, define splice sites size to 3, add loss of function (LOF), Nonsense mediated decay and OICR tags.
-"options": {
-   " -hgvs -noShiftHgvs -spliceSiteSize 3 -lof -oicr "}
-}</xmp><br><H3 id='annotations::exomiser'>annotations::exomiser</H3>Annotation process using Exomiser tool and options (see [Exomiser website documentation](https://www.sanger.ac.uk/tool/exomiser/)).<br>Examples: <xmp># Annotation with Exomiser, using database relse '2109', transcripts source as UCSC and a list of HPO terms.
+"options": " -hgvs -noShiftHgvs -spliceSiteSize 3 -lof -oicr "</xmp><br><H3 id='annotations::exomiser'>annotations::exomiser</H3>Annotation process using Exomiser tool and options (see [Exomiser website documentation](https://www.sanger.ac.uk/tool/exomiser/)).<br>Examples: <xmp># Annotation with Exomiser, using database relse '2109', transcripts source as UCSC and a list of HPO terms.
 "exomiser": {
    "release": "2109"
    "transcript_source": "refseq"
    "hpo": ['HP:0001156', 'HP:0001363', 'HP:0011304', 'HP:0010055']
 }</xmp><br><H4 id='annotations::exomiser::release'>annotations::exomiser::release</H4>Release of Exomiser database. This option replace the release variable in 'application.properties' file (see 'exomiser_application_properties' option). The release will be downloaded if it is not available locally. <br>Examples: <xmp># Annotation with release '2109' of Exomiser database.
-"release": "2109"</xmp><br>
+"release": "2109"</xmp><br><H3 id='annotations::options'>annotations::options</H3>Options for annotations, such as annotation strategy (skip if exists, update, append)<br>Examples: <xmp># Annotation with Parquet databases, with update annotation strategy.
+"options": {
+   "update": True
+}</xmp><br><H4 id='annotations::options::update'>annotations::options::update</H4>Update option for annotation (only for Parquet annotation). If True, annotation fields will be removed and re-annotated. These options will be applied to all annotation databases.<br>Examples: <xmp># Apply update on all annotation fields for all databases.
+"update": True</xmp><br><H4 id='annotations::options::append'>annotations::options::append</H4>Append option for annotation (only for Parquet annotation). If True, annotation fields will be annotated only if not annotation exists for the variant. These options will be applied to all annotation databases.<br>Examples: <xmp># Apply append on all annotation fields for all databases.
+"append": True</xmp><br>
diff --git a/docs/help.param.json b/docs/help.param.json
@@ -86,15 +86,6 @@
           "}",
           ""
         ]
-      },
-      "update": {
-        "__help": [
-          "Update option for Parquet annotation. If True, annotation fields will be updated if exists. If False, annotation fields will not change if it already exists. These options will be applied to all annotation databases."
-        ],
-        "__examples_code": [
-          "# Apply update on all annotation fields for all databases.",
-          "\"update\": True"
-        ]
       }
     },
     "bcftools": {
@@ -234,9 +225,7 @@
         ],
         "__examples_code": [
           "# Annotation with snpEff databases, with options to generate HGVS annotation, specify to not shift variants according to HGVS notation, define splice sites size to 3, add loss of function (LOF), Nonsense mediated decay and OICR tags.",
-          "\"options\": {",
-          "   \" -hgvs -noShiftHgvs -spliceSiteSize 3 -lof -oicr \"}",
-          "}"
+          "\"options\": \" -hgvs -noShiftHgvs -spliceSiteSize 3 -lof -oicr \""
         ]
       }
     },
@@ -261,6 +250,35 @@
           "\"release\": \"2109\""
         ]
       }
+    },
+    "options": {
+      "__help": [
+        "Options for annotations, such as annotation strategy (skip if exists, update, append)"
+      ],
+      "__examples_code": [
+        "# Annotation with Parquet databases, with update annotation strategy.",
+        "\"options\": {",
+        "   \"update\": True",
+        "}"
+      ],
+      "update": {
+        "__help": [
+          "Update option for annotation (only for Parquet annotation). If True, annotation fields will be removed and re-annotated. These options will be applied to all annotation databases."
+        ],
+        "__examples_code": [
+          "# Apply update on all annotation fields for all databases.",
+          "\"update\": True"
+        ]
+      },
+      "append": {
+        "__help": [
+          "Append option for annotation (only for Parquet annotation). If True, annotation fields will be annotated only if not annotation exists for the variant. These options will be applied to all annotation databases."
+        ],
+        "__examples_code": [
+          "# Apply append on all annotation fields for all databases.",
+          "\"append\": True"
+        ]
+      }
     }
   }
 }
diff --git a/docs/help.param.md b/docs/help.param.md
@@ -8,7 +8,6 @@ HOWARD Parameters JSON file defined parameters to process annotations, prioritiz
    - [annotations](#annotations)
       - [parquet](#annotationsparquet)
          - [annotations](#annotationsparquetannotations)
-         - [update](#annotationsparquetupdate)
       - [bcftools](#annotationsbcftools)
          - [annotations](#annotationsbcftoolsannotations)
       - [annovar](#annotationsannovar)
@@ -18,6 +17,9 @@ HOWARD Parameters JSON file defined parameters to process annotations, prioritiz
          - [options](#annotationssnpeffoptions)
       - [exomiser](#annotationsexomiser)
          - [release](#annotationsexomiserrelease)
+      - [options](#annotationsoptions)
+         - [update](#annotationsoptionsupdate)
+         - [append](#annotationsoptionsappend)
 
 
 ## annotations
@@ -112,16 +114,6 @@ Examples:
 
 ```
 
-#### annotations::parquet::update
-
-Update option for Parquet annotation. If True, annotation fields will be updated if exists. If False, annotation fields will not change if it already exists. These options will be applied to all annotation databases.
-
-Examples: 
-```
-# Apply update on all annotation fields for all databases.
-"update": True
-```
-
 ### annotations::bcftools
 
 Annotation process using BCFTools. Provide a list of database files and annotation fields.
@@ -270,9 +262,7 @@ String (as command line) of options available such as:
 Examples: 
 ```
 # Annotation with snpEff databases, with options to generate HGVS annotation, specify to not shift variants according to HGVS notation, define splice sites size to 3, add loss of function (LOF), Nonsense mediated decay and OICR tags.
-"options": {
-   " -hgvs -noShiftHgvs -spliceSiteSize 3 -lof -oicr "}
-}
+"options": " -hgvs -noShiftHgvs -spliceSiteSize 3 -lof -oicr "
 ```
 
 ### annotations::exomiser
@@ -299,3 +289,35 @@ Examples:
 "release": "2109"
 ```
 
+### annotations::options
+
+Options for annotations, such as annotation strategy (skip if exists, update, append)
+
+Examples: 
+```
+# Annotation with Parquet databases, with update annotation strategy.
+"options": {
+   "update": True
+}
+```
+
+#### annotations::options::update
+
+Update option for annotation (only for Parquet annotation). If True, annotation fields will be removed and re-annotated. These options will be applied to all annotation databases.
+
+Examples: 
+```
+# Apply update on all annotation fields for all databases.
+"update": True
+```
+
+#### annotations::options::append
+
+Append option for annotation (only for Parquet annotation). If True, annotation fields will be annotated only if not annotation exists for the variant. These options will be applied to all annotation databases.
+
+Examples: 
+```
+# Apply append on all annotation fields for all databases.
+"append": True
+```
+