Support several FAIRtracks versions at the same time (by dynamically loading all subschemas)

[sveinugu]

No description provided.

[sveinugu]

The augmentation service should support augmenting different versions of the FAIRtracks schema, depending on the @schema property of the supplied JSON document. This is needed by the recent changes in the FAIRtracks Makefile, for instance if is there is a need for bugfixes of previous versions of the standard. Also, new releases should not need a manual change of version information in the augmentation server. This at least means fixing the following:

• The FAIRtracks schemas are read from hardcoded URLs that include the version information. Instead generate the URLs based upon the @schema URL. Currently, the list of schemas are hardcoded, but it would be a great improvement if instead the full schema (including subschemas) was dynamically loaded from the top-level schema by a recursive search function. The following extension of getPathsToElement automatically recurse into referenced subschemas, and should work (I have tested it):

from copy import copy

def getPathsToElement(elName, url=None, data=None, path=[]):
    assert(url is not None or data is not None)

    if data is None:
        data = json.load(urllib.request.urlopen(url))

    if isinstance(data, dict):
        for key, val in data.items():
            newPath = copy(path)
            newUrl = url
            
            if key == '$ref':
                newUrl = '/'.join([url.rsplit('/', 1)[0], val])
                data = None
            else:
                newPath.append(key)
                data = val
                
            if key == elName:
                yield url, newPath, val
            else:
                for _ in getPathsToElement(elName, newUrl, data, newPath):
                    yield _
    elif isinstance(data, list):
        for i, item in enumerate(data):
            for _ in getPathsToElement(elName, url, item, path + [i]):
                yield _

This allows for code like the following, which should fix #15:

targetPropDict = {}

for url, path, val in getPathsToElement('required', topSchemaUrl):
    if all(_ in path for _ in ['allOf', 'target']):
        targetPropDict[(url, tuple(path[:2]))] = val

for url, path, val in getPathsToElement('const', topSchemaUrl):
    targetProp = targetPropDict.get((url, tuple(path[:2])))
    if targetProp:
        print(val, targetProp)

The function can also be used to traverse the input JSON data by setting url=None.

Implementing the above changes allows for simplifying much of the rest of the code, e.g. for fetching ontology URLs (with no need to hardcode hacks for reused schemas like currently with phenotype). But it requires quite a bit of rewrite.

• Add a possibility to supply the json schemas in the augment request and use these instead of downloading them from github