Updates primarily around web users, but with some API updates

SACGF · Aug 5, 2019 · e1cf7db · e1cf7db
1 parent f39c269
commit e1cf7db
Show file tree

Hide file tree

Showing 24 changed files with 154 additions and 56 deletions.
diff --git a/docs/index.rst b/docs/index.rst
@@ -1,15 +1,32 @@
-Welcome to Shariant Integration documentation
+Welcome to Shariant documentation
 ==================================================
 
+This documentation provides information both for web users or the product, and developers who are connecting to Shariant's API.
+
 .. toctree::
    :maxdepth: 1
-   :caption: FAQ:
+   :caption: What is Shariant
 
    faq/faq
 
 .. toctree::
    :maxdepth: 1
-   :caption: Technical Basics:
+   :caption: How to Navigate Shariant
+
+   site/web_interface_overview
+   site/classification_dashboard
+   site/classification_diffs
+   site/classification_discordance
+   site/classification_flags
+   site/classification_form
+   site/classification_listing
+
+If you are working on the technical connection to Sharaint
+----------------------------------------------------------
+
+.. toctree::
+   :maxdepth: 1
+   :caption: Technical Basics
 
    integration/basics/overview
    integration/basics/getting_started
@@ -20,35 +37,17 @@ Welcome to Shariant Integration documentation
 
 .. toctree::
    :maxdepth: 1
-   :caption: Evidence Keys:
+   :caption: Evidence Keys
 
    integration/evidence_keys/overview
    integration/evidence_keys/types
 
 .. toctree::
    :maxdepth: 1
-   :caption: API:
+   :caption: API
 
    integration/api/philosophy
    integration/api/authentication
    integration/api/classification_post
    integration/api/classification_export
-   integration/api/classification_json
-
-.. toctree::
-   :maxdepth: 1
-   :caption: Web Interface Classifications:
-
-   site/classification_dashboard
-   site/classification_diffs
-   site/classification_discordance
-   site/classification_flags
-   site/classification_form
-   site/classification_listing
-
-Indices and tables
-==================
-
-* :ref:`genindex`
-* :ref:`modindex`
-* :ref:`search`
+   integration/api/classification_json
diff --git a/docs/integration/api/authentication.md b/docs/integration/api/authentication.md
@@ -2,7 +2,7 @@
 
 The Shariant API will require authentication to perform any operation. The authentication is currently provided through OAuth2 via Keycloak.
 
-The OAuth URL is
+The OAuth URL (for production and test) is
 `https://shariant.org.au/auth/realms/agha/protocol/openid-connect/token`
 provide a client_id of
 `shariant-client-tool`
@@ -31,9 +31,8 @@ def ping() {
 	)
 	auth = OAuth2(client_id = client_id, token = token)
 
-	r = requests.get('https://shariant.org.au/variantclassification/api/evidence_keys', auth=auth)
+	r = requests.get('https://shariant.org.au:444/variantclassification/api/evidence_keys', auth=auth)
 }
 ```
 
-Note that any classifications you make through the API will be assigned to the username used here by default.
-Records "owner" propert
+Note that any classifications you make through the API will be assigned to the user used to authenticate against the API by default.
diff --git a/docs/integration/api/classification_json.md b/docs/integration/api/classification_json.md
@@ -2,26 +2,36 @@
 
 An example of the result of a GET or POST below
 
-THIS EXAMPLE IS OUT OF DATE AND USES v1 OF THE API.
-VERSION 2 MAKES THE OUTPUT MIRROR THE IMPORT FORMAT, MOVING MOST OF THE CALCULATED FIELDS TO A "meta" object.
-
 ```json
 {
     "id": 31,
-    "lab_record_id": "77",
-    "institution_name": "SA Pathology",
-    "lab_id": "sa_pathology/unit_1",
-    "lab_name": "SA Pathology Unit 1",
-    "title": "SA Pathology Unit 1 31",
-    "user": "varadmin",
-    "version": 1544864360.852882,
-    "can_write": true,
-    "can_write_latest": true,
-    "flag_collection": 13,
-    "has_changes": true,
-    "last_edited": 1553238219.242146,
-    "variant_id": 483370,
-    "variant": "11:103048382 T>A",
+    "meta": {
+        "lab_record_id": "77",
+        "institution_name": "SA Pathology",
+        "lab_id": "sa_pathology/unit_1",
+        "lab_name": "SA Pathology Unit 1",
+        "title": "SA Pathology Unit 1 31",
+        "user": "varadmin",
+        "version": 1544864360.852882,
+        "can_write": true,
+        "can_write_latest": true,
+        "flag_collection": 13,
+        "has_changes": true,
+        "last_edited": 1553238219.242146
+    },
+    "allele": {
+        "clingen_allele_id": "CA396457842",
+        "genome_builds": {
+            "GRCh37": {
+                "variant_coordinate": "16:68842599 A>G",
+                "g_hgvs": "NC_000016.9:g.68842599A>G",
+                "c_hgvs": "NM_004360.4(CDH1):c.535A>G",
+                "origin": "Imported as this build",
+                "variant_id": 535913
+            }
+        }
+    },
+    "publish": "logged_in_users",
     "data": {
         "age": {
             "value": "76"

diff --git a/docs/integration/api/classification_post.md b/docs/integration/api/classification_post.md
@@ -2,7 +2,7 @@
 All operations against a variant classification submission from your lab can be performed against a single end point.
 
 That end point being
-https://shariant.org.au/variantclassification/api/classifications/record/
+https://shariant.org.au/variantclassification/api/v2/classifications/record/
 
 Ensure that any posts there are passing the required authentication data and is sending data as ContentType "application/json"
 

diff --git a/docs/integration/api/philosophy.md b/docs/integration/api/philosophy.md
@@ -6,4 +6,7 @@ There are some fundamental principals that we're currently applying to the API
 * As long as the API submission is well formatted JSON, nearly everything uploaded will create a new (or alter an existing) variation classification record. The record itself may be marked with errors but they can be fixed with subsequent API calls or interaction with the web UI.
 * There will be a level of normalisation applied to submitted data, case insensitive "true", "T", 1 will be converted to `true` - and "false", "F" and 0 to `false` for boolean fields, drop down field values will have their case corrected. The level of normalisation will be limited as to reduce unexpected behaviour.
 * We'll be mainly accepting free form text, so citing of publications or other resources will be achieved by parsing through the text looking for patterns, such as "PMID: XXXX" rather than requiring special structure in the submission.
-* Under no circumstances should patient identifiable data be uploaded.
+* Under no circumstances should patient identifiable data be uploaded.
+
+Please note there is a test server https://shariant.org.au:444/
+The same privacy rules apply to the test server (e.g. even when testing ensure personal patient details do not get uploaded) but you are free to send dummy data to the test server.
diff --git a/docs/integration/basics/getting_started.md b/docs/integration/basics/getting_started.md
@@ -65,4 +65,4 @@ Assigning resposible users may cause less coordination work for your team, thoug
 ## Step 5 : How best can your system integrate data from Shariant
 
 Shariant provides an API for bulk downloading of classifications.
-Currently we only provide classifications in our own JSON format, but on the roadmap is support for more export options such as VCF. It is expected that some systems will need custom solutions here, so some work may get done as the requirements come in.
+Currently we provide classifications in our own JSON format, CSV, MVL, VCF. It is expected that most systems will be able to use these formats but some may require a custom solution.
diff --git a/docs/integration/basics/sharing.md b/docs/integration/basics/sharing.md
@@ -4,22 +4,20 @@ Shariant has the following concepts for handling who has access to what
 
 |Body|Meaning|
 |----|-------|
-|Organisation|Aka Institution (eg SA Pathology, VCGS) |
 |Lab|Lab in an organisation (eg SA Pathology’s Familial Cancer, Frome Road) |
+|Organisation|Aka Institution (eg SA Pathology, VCGS) |
 |User|Each lab member should have their own login. The sync tool will have a separate account. |
 |Variant Classification|A variant classification will be owned by a user and a lab |
 
 A user can belong to multiple labs, though typically a user will only belong to one. Please contact us when a staff member leaves your lab and we can disable access to your lab's records.
 
 Variant Classifications can be seen in two modes.
 
-* The live editable copy (there is only 1 of these)
+* The live editable copy
 * A read-only version shared at a given point in time
 
-There can be multiple read-only versions for each live editable classification.
-
 If you or someone from your lab created a variant classification, you will be dealing with editable copy.
-If someone from outside your lab shares a record with you, you will be dealing with the specific version that they deemed to share. If they make changes and share it again, you will then have access to the new version. The same applies when you share records with them
+If someone from outside your lab shares a record with Shariant, you will be dealing with the latest published version of that record. If they make changes and share it again, you will then have access to the new version. The same applies when you share records with Shariant.
 
 Users with access to the editable version can elect to share the record in its current state as long as there are no outstanding validation errors. This will give other users read only access to the data as it is when the publish action was performed.
 
@@ -34,4 +32,4 @@ Sharing can be done at several levels. Each level encompasses the level before i
 
 See the Sharing section in the API for information on how to utilise these share levels.
 
-The purpose of Shariant is to share records. The lower share levels are intended for records that are awaiting review or more information - not as a permanent half sharing state, and the system will automatically share records to other labs within Shariant after 6 months to ensure progress down the Sharing lifecycle.
+Just a reminder that the purpose of Shariant is to share records.
diff --git a/docs/site/classification_form.md b/docs/site/classification_form.md
@@ -1,3 +1,53 @@
 # Classification Form
 
-Form
+The Classification Web Form can be used to create and edit classifications directly within Shariant.
+
+However it is expeced that classifications be created in your own classification system, and automatically synced to Shariant.
+That said, the Classification Web Form still provides some utility.
+
+The web form provides a good readable version of Shariant's version of the web form.
+
+### View
+
+![](images/classification_form.png)
+
+To quickly see all fields that have values for a classification, enter "*" into the filter box at the top of the classification.
+
+If you're looking at one of your own records, if a value was provided by your curation system to Shariant connector, it will be read only. If there are values that are never uploaded, you can edit them directly on the form. (This would be a very time consuming process to do for all classifications so it's assumed it will only be done in special cases like discordances).
+
+Note that for other users to see changes you have to perform a submit or share (see Actions).
+
+You can also enter notes next to fields, be warned that if your connector to Shariant sends up notes, your notes might get overriden.
+
+### Identify Errors
+
+A record might not be shared as there are outstanding validation errors. In the Messages box on the form it will list any errors. If possible fix those errors in your curation system and then they should be fixed on the next sync.
+
+### Change History / Diff
+
+Each version of a record published in Shariant is recorded, by clicking on "Compare historical versions of this record".
+
+If there are other classifications for the same variant, there will be a link to compare them there too.
+
+### Actions
+
+![](images/classification_form_actions.png)
+
+At the bottom of the form there will be a list of action buttons.
+
+The Tick icon re-submits the classification at its current change level. For any manual changes to be seen, this button will need to be ticked.
+
+Next to it is a Share button that allows you to increase the scope of who can see the classification.
+Important, increasing the Share level is not un-doable.
+The share levels are
+* Just your lab
+* Anyone within your organisation (if your organisation has multiple labs)
+* All Shariant Users
+* 3rd Party Databases (this will allow us to upload the record to Clinvar at a later date)
+
+Delete / Withdraw
+
+If the classification has only been shared at the lab or organisation level, you are able to perform a hard delete on the record.
+If it has been shared, instead you have the option to "withdraw". This will remove the record from most listings and search results, but will not remove it from any Discordance Reports that it had been involved in (it will no longer be a part of discordance calculations).
+
+When a record has been withdrawn it can be unwithdrawn by clicking the same button (it should look like a rubbish bin with a raised lid now).
diff --git a/docs/site/classification_listing.md b/docs/site/classification_listing.md
@@ -1,3 +1,33 @@
 # Classification Listing
 
-TODO
+(!) The classification listing page has not yet been optimised for Shariant use.
+You may find it not optimal to use currently.
+
+### Records Syncing Down
+
+Hopefully as well as uploading to Shariant, your system is also configured to download records from Shariant and import them into your curation system.
+If this is the case it is recommended that you search for variants within your own classification system as you are probably well versed in its interface.
+
+### Using the Shariant Classification Listing
+
+The classification listing shows classifications alongside a pie chart that shows overall clinical significance of all records stored.
+Also present is a stacked bar chart that shows you how much of each kind clinical significance is stored for each gene.
+
+Clicking on any of these graph segments filters the results to that significance or gene/significance combination.
+
+There is also a Mine checkbox that will only show records if you are recorded as the "owner" of the record.
+
+There is a Gene filter where you can search using the Ensembl Gene ID or gene name (but a single ensemble gene must be seleced)
+
+The "Simple Filter" provides limited filtering options. It is intended that this filter interface be replaced in future.
+
+Otherwise you can navigate through the table of results.
+
+### Export
+
+You can also export the single record as CSV, a preview of the Clinvar format or as a report.
+(The report does require that your lab has a report template pre-configured.)
+
+### Literature Citations
+
+Any PMID references in the form of PMID:123456 from anywhere within the classification will be summed together and listed at the bottom of the classification.
diff --git a/docs/site/images/analysis.png b/docs/site/images/analysis.png
diff --git a/docs/site/images/checklist.png b/docs/site/images/checklist.png
diff --git a/docs/site/images/classification_form.png b/docs/site/images/classification_form.png
diff --git a/docs/site/images/classification_form_actions.png b/docs/site/images/classification_form_actions.png
diff --git a/docs/site/images/discordance.png b/docs/site/images/discordance.png
diff --git a/docs/site/images/discordance_report.png b/docs/site/images/discordance_report.png
diff --git a/docs/site/images/flag-not-found.png b/docs/site/images/flag-not-found.png
diff --git a/docs/site/images/flag_dialog.png b/docs/site/images/flag_dialog.png
diff --git a/docs/site/images/lightbulb.png b/docs/site/images/lightbulb.png
diff --git a/docs/site/images/lock.png b/docs/site/images/lock.png
diff --git a/docs/site/images/not-found.png b/docs/site/images/not-found.png
diff --git a/docs/site/images/outstanding_edits.png b/docs/site/images/outstanding_edits.png
diff --git a/docs/site/images/trash.png b/docs/site/images/trash.png
diff --git a/docs/site/images/work.png b/docs/site/images/work.png
diff --git a/docs/site/web_interface_overview.md b/docs/site/web_interface_overview.md
@@ -0,0 +1,9 @@
+# Web Interface Overview
+
+It is intended that the majority of classification data from your lab be automatically synced to Shariant. Shariant is not intended to be a place for large manual data entry. Conversely classifications in Shariant should be automatically downloaded to your curation system.
+
+Given this, there are still plenty of uses for the web interface.
+
+* [Identify and fix issues with classifications from your lab.](classification_dashboard)
+* [Resolve any classification discordances.](classification_discordance)
+* [Manually find classifications.](classification_listing)