SNOMED Commit API seems to be too coarse-grained

[psiotwo]

After successful import of SNOMED Intl FULL (20190131 release), and then I wanted to get the change history for a concept (incl. relationships where from/to is the chosen concept and descriptions of that concept). Currently the only option seems to me to use the /commits API to get all the commits related to the concept, e.g.:

/snomed-ct/v3/commits?affectedComponentId=61462000

This results in a list of commits, containing e.g.

 {
            "id": "7ca1c4da-a045-4f7c-a6e0-f632d5243b06",
            "branch": "MAIN",
            "author": "System",
            "comment": "Imported components from 2015-01-31",
            "timestamp": 1574255944125
},

Since the commit wraps the whole import, getting its details by

/snomed-ct/v3/commits/7ca1c4da-a045-4f7c-a6e0-f632d5243b06?expand=details()

results in a huge output which also takes a minute or more to get. Unless I missed some parameter for this call that would allow me to filter by concept SCTID on the server side (in that case I would first get the current descriptions and relationships for the concept and then get their details across all the commits).

Any hint how to use the API efficiently for imported SNOMED components would be appreciated!

[cmark]

Hi @psiotwo,

Yes, this is a limitation of the current commits API, unfortunately.
Usually what we do to show the history of a single concept is fetch everything in one go:

GET /snomed-ct/v3/commits?affectedComponentId=61462000&expand=details()

The details() expand will actually use the affectedComponentId filter to limit the details object to the requested component ID and all its related component changes (description/relationship/member related commits will be returned when you request the history of the concept, you don't have to make explicit requests for them).
So basically that is the only request you need to make to get the entire history of the concept.

Additionally to make life easier and support individual commits, where the details() expansion is defined, I'll improve the API with the following two changes:

Filter by id option will be added to GET /commits API:

`GET /snomed-ct/v3/commits?id=7ca1c4da-a045-4f7c-a6e0-f632d5243b06&affectedComponentId=61462000&expand=details()`

Also, affectedComponentId details filter will be added to details() expand:

`GET /snomed-ct/v3/commits/7ca1c4da-a045-4f7c-a6e0-f632d5243b06?expand=details(affectedComponentId:61462000)`

These changes will be available soon on the 7.x branch and will be released in 7.4.0 end of February.
I hope these changes make sense to you if you have any feedback or comment, let us know.

Cheers,
Mark

[psiotwo]

Hi @cmark ,
yes, thanks - sounds perfect!
Petr

[psiotwo]

Hi @cmark,

thanks, just tested.

Maybe I misinterpreted your intent. When I issue the first request
GET /snomed-ct/v3/commits?id=7ca1c4da-a045-4f7c-a6e0-f632d5243b06&affectedComponentId=61462000&expand=details()

I still get the whole commit in the result. What I would need is to filter only those items in the commit which refer to the concept (or related descriptions/relationships), e.g. by filtering on items/detail/object:61462000. Maybe I just don't know how to correctly construct the expand parameter (similarly to the affectedComponentId:61462000 mentioned below).

Also after issuing
GET /snomed-ct/v3/commits/7ca1c4da-a045-4f7c-a6e0-f632d5243b06?expand=details(affectedComponentId:61462000)

I get:

{
    "status": 400,
    "code": 0,
    "message": "Expected type 'String' is not valid for the value '61462000(Integer)' returned for the key 'affectedComponentId'",
    "errorCode": 0,
    "statusCode": 400
}

Petr

[cmark]

Hi @psiotwo,

No, you did not, you did everything perfectly, the example I gave you was incorrect.
Inside a nested object expansion, like in details(), we treat everything as JSON and parse it accordingly. So to be able to parse the affectedComponentId as a String value, you need to wrap it in "61462000", so the correct example would look like this:

GET /snomed-ct/v3/commits/7ca1c4da-a045-4f7c-a6e0-f632d5243b06?expand=details(affectedComponentId:"61462000")

Let me know if this works for you.

Cheers,
Mark

[psiotwo]

Hi @cmark ,

thanks for hint. First it seemed to me it worked :-). But, then tried for a commit which corresponds to the 2002-07-31 release import of Snomed CT Intl. and got:

{
    "id": "9a463a2e-e718-4f6d-890a-34a20889401f",
    "branch": "MAIN",
    "author": "System",
    "comment": "Imported components from 2002-07-31",
    "timestamp": 1580314281451,
    "details": {
        ...
        "limit": 119031,
        "total": 119031
    }
}

no matter whether I specify affectedComponentId or not, i.e. both of the following calls return a result set with the same "total".

/snomed-ct/v3/commits/9a463a2e-e718-4f6d-890a-34a20889401f?expand=details()

and

/snomed-ct/v3/commits/9a463a2e-e718-4f6d-890a-34a20889401f?expand=details(affectedComponentId:"61462000")

Petr

[psiotwo]

Hi @cmark ,

I was testing stuff on wrong deployment and can confirm that everything works with me exectly as you described.

I tried even a more "bold query" to get all the details across all the commits in one run:

/snomed-ct/v3/commits?expand=details(affectedComponentId:"239783001")

It is working fine, but it returns also commits which don't reference this component at all, e.g.

{
            "id": "0578874b-c45e-4565-ac02-76aa73ee5495",
            "branch": "MAIN",
            "author": "System",
            "comment": "Imported components from 2002-01-31",
            "timestamp": 1574253114834,
            "details": {
                "items": [],
                "limit": 0,
                "total": 0
            }
},

Does it make sense to filter out these commits from the response?

Cheers,
Petr

[cmark]

Hi @cmark ,

thanks for hint. First it seemed to me it worked :-). But, then tried for a commit which corresponds to the 2002-07-31 release import of Snomed CT Intl. and got:

{
    "id": "9a463a2e-e718-4f6d-890a-34a20889401f",
    "branch": "MAIN",
    "author": "System",
    "comment": "Imported components from 2002-07-31",
    "timestamp": 1580314281451,
    "details": {
        ...
        "limit": 119031,
        "total": 119031
    }
}

no matter whether I specify affectedComponentId or not, i.e. both of the following calls return a result set with the same "total".

/snomed-ct/v3/commits/9a463a2e-e718-4f6d-890a-34a20889401f?expand=details()

and

/snomed-ct/v3/commits/9a463a2e-e718-4f6d-890a-34a20889401f?expand=details(affectedComponentId:"61462000")

Petr

Hi @psiotwo,

The filtering bug has been fixed and the details() expand with an affectedComponentId filter will now produce the correct numbers.

Regarding the other issue you have reported, you only restrict the details() expansion to the given affectedComponentId, but you still need all commits because you did not filter them by any means.
From an API perspective, this works as expected and how it should.
If you would like to restrict the commits to a given component then please use the affectedComponentId filter.

Cheers,
Mark