Outdated version in VarFish ClinVar export

[ahujameg]

Describe the bug
The XSD schema version for clinvar export used in VarFish is 1.6. The latest schema version in ClinVar is 1.7 where enumeration for Named Protein Variant is added to the list of Trait Types.

To Reproduce
Steps to reproduce the behavior:

1. Open the XSD schema used in Varfish ClinVar export.
2. Check the schema version and other differences. It is 1.6
3. Check and compare the schema version from the Clinvar site

Expected behavior
The ClinVar Export mechanism in Varfish should use the latest version from ClinVar which is 1.7

Desktop (please complete the following information):

• OS: Linux
• Browser chrome, safari
• Version 22

[holtgrewe]

@ahujameg confirmed; could you list the concrete issues (what parts of the XML currently have to be changed manually for the workaround) and sketch your proposed changes?

[ahujameg]

Root Cause Analysis

A previous XML schema was used for the generation of data which is not accepted any more. Apparently, Clinvar invalidates the old data format once the new one is created. We will (only) need to adjust the XML generation code.

Resolution Proposal

Add the latest XSD changes used for ClinVar Export and use this new XSD wherever it is referenced.

Resolution Proposal Summary

Affected Components
VarFish Server (only)

Affected Modules/Files
clinvar_submission_1.6.xsd, clinvar_xml.py, views_ajax.py

Required Architectural Changes
none

Resolution Sketch

1. Rename clinvar_submission_1.6.xsd to clinvar_submission_1.7.xsd

   <xs:schema version="1.7; May 22, 2020" xmlns:xs="http://www.w3.org/2001/XMLSchema">
       <xs:simpleType name="Traittypelist">
           <xs:restriction base="xs:string">
               <xs:enumeration value="Disease"/>
               <xs:enumeration value="DrugResponse"/>
   		   	...
       		...
               <xs:enumeration value="NamedProteinVariant"/>
   

2. clinvar_xml.py
   Update the URL in line number 7 and rename the variable XSD_URL_1_6 to XSD_URL_1_7
   XSD_URL_1_7 = "https://ftp.ncbi.nlm.nih.gov/pub/clinvar/xsd_submission/clinvar_submission_1.7.xsd"
3. views_ajax.py
   Use the renamed variable XSD_URL_1_7 in line 67:

   XSD_CONTENTS = {
       XSD_URL_1_7: _read(XSD_URL_1_7.rsplit("/", 1)[1]),
   }
   

[holtgrewe]

@ahujameg This looks good to me. Can you restructure your comment with headings "Root Cause Analysis" and "Proposed Change". You can use the following as a template: #234

Feel free to just put "A previous XML schema was used for the generation of data which is not accepted any more. Apparently, Clinvar invalidates the old data format once the new one is related. We will (only) need to adjust the XML generation code." as root cause analysis results. Afterwards, please create a branch 226-update-clinvar-xml-export or similar in a fork of yours and create a draft merge request. It would be helpful if you could add a test that exposes the problem for this (I believe) untested part of VarFish (a test would not have caught this problem anyway as the issue occurs with an external service).

[holtgrewe]

@ahujameg looks good now. Please note that I'm not notified via Github of any edits to a comment. Please @-mention me next time so we can get this forward more quickly. Please start a branch in your fork starting with 226- and having a short meaningful name, create a draft pull request, and assign me for review once you are done.

[ahujameg]

@holtgrewe

When tying to push the new branch (before creating a draft pull request), there is a permission denied error.

[ahujameg]

@holtgrewe
Push worked now, so I have created a draft pull request (#270).
The assignee or reviewers fields are not editable for me.