Add support for concrete types, for indexing and for search

[rlustemberg]

I'm trying to run an ECL query on a terminology server using the Argentina edition of SNOMED CT. The query is meant to retrieve concepts with specific attributes, but it's not returning the expected results.
Here's the ECL query I'm using:

<781025005: 763032000 = 732936001, 1142135004 = #250

Concept ID 781025005 (Product containing metamizole)
Attribute 763032000 (Has dose form) with value 732936001 (Oral tablet)
Attribute 1142135004 (Has presentation strength numerator value) with value #250

The presentation strength refinement is the one not working. Using a wildcard returns an empty array, using a value (#250) returns 404. The correct result count in the case of #250 is 11 results.
You can test here for the correct output https://browser.ihtsdotools.org/?perspective=full&conceptId1=404684003&edition=MAIN/SNOMEDCT-ES/SNOMEDCT-AR/2022-11-30&release=&languages=es,en

Thanks in advance. I'm looking closely into your commits to be of more help in future issues and learn some LISP along the way. Thank you again for your awesome piece of software

[wardle]

Hi. Thanks. The error message is not particularly helpful - this needs fixing.

What it should have said is that expressions containing concrete refinements are not yet supported. At least then you'd know it wasn't something you were doing wrong!

The last time I looked to implement, I found the UK distribution did not include any concepts with concrete values, so haven't prioritised this. I see the main International release does have a small number of concrete values (32k).

These concrete data types may become very useful for me in supporting the UK's dictionary of medicines - but this hasn't yet been released so I am afraid I haven't prioritised this. That said, it would be quite straightforward to support, but would need an update to indexing - for both store and search - in order to properly deal with concrete values.

[wardle]

Created new issue #48 to capture poor quality error message for users of the HTTP server, given that the underlying Hermes library is appropriately returning detailed diagnostics about the ECL parsing problem, including line number etc.

[rlustemberg]

Thanks a lot for your response! In the meantime we'll just do a search with s=250&constraint= <781025005: 763032000 = 732936001, which does the job good enough

[wardle]

I've created a new release. Would be interesting to know whether this works for your original query now - I have tested against the limited set of concrete values that can be found in the International edition, but the UK edition does not include them yet. I have not implemented search for string or boolean concrete values, as I haven't seen these 'in the wild' at all, and don't see any use at the moment. Numeric values do have a clear use however.

[wardle]

Hi @rlustemberg - can I double check that this is now working as expected? Recent releases should now include being able to index and search concrete values. I don't have access to the Argentina distribution, only the International and UK editions, so my testing of concrete values has been limited to synthetic data and what is in the International edition. Notably, I have not seen any string or boolean type concrete values.