Implementation of $apply API

[fredhersch]

User story:
As an implementer I can use the $apply operation to generate a CarePlan for a patient given a PlanDefinition

Background
CQL (Clinical Quality Language) is the HL7 standard for providing logic execution for decision support and clinical quality measures.

Our initial goal with the CQL Library will be to demonstrate over time the ability to:

• Generate a MeasureReport via $measure-evaluate operation
• Load and execute a custom CQL library (Library resource) for decision logic
• Transform a PlanDefinition to a CarePlan using $apply operation

[jingtang10]

dependencies:

• Support JSON as a serialization format for ModelInfo cqframework/clinical_quality_language#640
• Annotation serialization with JXSON cqframework/clinical_quality_language#523

[maimoonak]

Few Observations during code exploration

• CRUD in CQL, does fhir spec allow it or not?
• CRUD in Measure, does fhir spec allowsit or not?
• Library Processor L213 -> if (!url.contains("/Library/"))
• Library name RuleFilter
• ELM+JSON LIbrary works with CQL. XML throws Serializer exception
• Translating Library matcher translatorOptionsMatch needs replacement with matcher method
• The CQL Builder with default constructor does not initialize properly with library loader hence missing libraries on runtime
• libraryCache needs to be carefully implemented, currently alot of initialization happens with in classes privately
• The FHIRDal i.e. CQL would be loading data on demand. This restricts us running CQL before saving entities (For example we just want to test questionnaire, extraction, and CQL without persisting entities)
• ExpressionEvaluator runs expression as library but in resources no content / library has been provided (removing expressions make testcase pass)
  libraryProcessor.evaluate(new VersionedIdentifier().withId("expression").withVersion("1.0.0"), subject,

Plz check commit here. These changes were required to run the library loaders. The test does not pass though due to expression not added as library (Not sure if I am missing anything here)

@f-odhiambo @pld @jingtang10 @dubdabasoduba

[f-odhiambo]

Seem like we have the following

Input

val cp = fhirOperator.generateCarePlan(
      planDefinitionId = “plandefinition-RuleFilters-1.0.0”,
      patientId = “Reportable”,
      encounterId = “reportable-encounter”
    )

and gettging the following output based on @maimoonak updates

Output
{"resourceType":"CarePlan","contained":[{"resourceType":"RequestGroup","id":"3ddb8524-345a-4477-9589-4144b71ff88c","status":"draft","intent":"proposal"}],"instantiatesCanonical":["plandefinition-RuleFilters-1.0.0"],"status":"draft","subject":{"reference":"Reportable"},"encounter":{"reference":"reportable-encounter"},"activity":[{"reference":{"reference":"#3ddb8524-345a-4477-9589-4144b71ff88c"}}]}

Are you getting the same @jingtang10?

CC @pld @mberg

[pld]

Can we re-open this issue? I don't believe the above issues raised by @maimoonak and @f-odhiambo were addressed and tested. But maybe I am missing those tests, can you link to them?

[jingtang10]

Thanks @pld for raising this. In the call earlier today we discussed this and @maimoonak will share the changes she has made and raise a draft pr against this issue.

also would love to get input from jp and vitor (sent them invite to the repo).

[Tarun-Bhardwaj]

@f-odhiambo , though the associated PR #1195 is merged, what would it need to close this issue?

[f-odhiambo]

I think we can track the $apply issues based on @vitorpamplona updates. I am ok closing this issue. Will update and coordinate with @maimoonak on where we can push the subsequent updates CC @Tarun-Bhardwaj

[jingtang10]

Few Observations during code exploration

• CRUD in CQL, does fhir spec allow it or not?
• CRUD in Measure, does fhir spec allowsit or not?
• Library Processor L213 -> if (!url.contains("/Library/"))
• Library name RuleFilter
• ELM+JSON LIbrary works with CQL. XML throws Serializer exception
• Translating Library matcher translatorOptionsMatch needs replacement with matcher method
• The CQL Builder with default constructor does not initialize properly with library loader hence missing libraries on runtime
• libraryCache needs to be carefully implemented, currently alot of initialization happens with in classes privately
• The FHIRDal i.e. CQL would be loading data on demand. This restricts us running CQL before saving entities (For example we just want to test questionnaire, extraction, and CQL without persisting entities)
• ExpressionEvaluator runs expression as library but in resources no content / library has been provided (removing expressions make testcase pass)
  libraryProcessor.evaluate(new VersionedIdentifier().withId("expression").withVersion("1.0.0"), subject,

Plz check commit here. These changes were required to run the library loaders. The test does not pass though due to expression not added as library (Not sure if I am missing anything here)

@f-odhiambo @pld @jingtang10 @dubdabasoduba

Let's close this issue once we address everything in @maimoonak 's commit in her comment above. @maimoonak please close this issue or raise a PR if necessary.

[maimoonak]

Current code has $apply unit test ignored hence we never knew that it is not working.
I have removed the Ignore and now test fails. The PR #1471 resolve some of the issues we need to address to run the test. I could not find a solution to last one. Looks like there is a progress from previous state but it still does not work. The step by step exception vs solution are below

the RuleFilters-1.0.0-bundle.json file might be collapsed into PR so plz expand to see the changes

1- Issue in cql library where it expects that the URL would be from server, however, by fhir definition it can be Canonical Local URL. Change PlanDefinition prefix / to -> "library": [ "Library/library-RuleFilters-1.0.0" ],

Invalid resource type for determining library version identifier: Library
java.lang.IllegalArgumentException: Invalid resource type for determining library version identifier: Library
	at org.opencds.cqf.cql.evaluator.library.LibraryProcessor.getVersionedIdentifer(LibraryProcessor.java:214)
	at org.opencds.cqf.cql.evaluator.library.LibraryProcessor.evaluate(LibraryProcessor.java:93)

2- The cql loader internally work (compare and search) on library name so change the Library to correct reference i.e. "name": "RuleFilters" -> "name": "library-RuleFilters-1.0.0"

java.lang.NullPointerException
	at com.google.android.fhir.workflow.FhirEngineLibraryContentProvider.getLibrary(FhirEngineLibraryContentProvider.kt:30)
	at org.opencds.cqf.cql.evaluator.cql2elm.content.fhir.BaseFhirLibraryContentProvider.getLibraryContent(BaseFhirLibraryContentProvider.java:35)
	at org.opencds.cqf.cql.evaluator.engine.execution.TranslatingLibraryLoader.getLibraryContent(TranslatingLibraryLoader.java:112)

3- The expression in condition matchers throw exception -> Remove null check on FhirEngineLibraryContentProvider as internally it creates library and adds it to InMemoryLibraryLoader and let cql engine search into all library providers

java.lang.NullPointerException
	at com.google.android.fhir.workflow.FhirEngineLibraryContentProvider.getLibrary(FhirEngineLibraryContentProvider.kt:30)
	at org.opencds.cqf.cql.evaluator.cql2elm.content.fhir.BaseFhirLibraryContentProvider.getLibraryContent(BaseFhirLibraryContentProvider.java:35)
	at org.opencds.cqf.cql.evaluator.engine.execution.TranslatingLibraryLoader.getLibraryContent(TranslatingLibraryLoader.java:112)

4- For expression as library org.opencds.cqf.cql.evaluator.expression.ExpressionEvaluator#getFhirVersion hardcodes fhir version 4.0.1 hence throw exception -> library added to Bundle but also needed change in FhirEngineLibraryContentProvider.libs to make key a Pair of name,version to pick correct library when needed

Translation of library expression failed with the following message: Library FHIRHelpers was included as version 4.0.1, but version 4.0.0 of the library was found.
org.opencds.cqf.cql.engine.exception.CqlException: Translation of library expression failed with the following message: Library FHIRHelpers was included as version 4.0.1, but version 4.0.0 of the library was found.
	at org.opencds.cqf.cql.evaluator.engine.execution.TranslatingLibraryLoader.translate(TranslatingLibraryLoader.java:134)
	at org.opencds.cqf.cql.evaluator.engine.execution.TranslatingLibraryLoader.load(TranslatingLibraryLoader.java:78)
	at org.opencds.cqf.cql.evaluator.engine.execution.CacheAwareLibraryLoaderDecorator.load(CacheAwareLibraryLoaderDecorator.java:49)

5- FhirEngineLibraryContentProvider updated to check for name and version if given, otherwise if version is null just return matching library by name

Unable translate library library-RuleFilters-1.0.0
org.opencds.cqf.cql.engine.exception.CqlException: Unable translate library library-RuleFilters-1.0.0
	at org.opencds.cqf.cql.evaluator.engine.execution.TranslatingLibraryLoader.translate(TranslatingLibraryLoader.java:128)
	at org.opencds.cqf.cql.evaluator.engine.execution.TranslatingLibraryLoader.load(TranslatingLibraryLoader.java:78)

6- Unresolved

Translation of library expression failed with the following message: Could not resolve identifier encounter in the current library.
org.opencds.cqf.cql.engine.exception.CqlException: Translation of library expression failed with the following message: Could not resolve identifier encounter in the current library.
	at org.opencds.cqf.cql.evaluator.engine.execution.TranslatingLibraryLoader.translate(TranslatingLibraryLoader.java:134)
	at org.opencds.cqf.cql.evaluator.engine.execution.TranslatingLibraryLoader.load(TranslatingLibraryLoader.java:78)
	at org.opencds.cqf.cql.evaluator.engine.execution.CacheAwareLibraryLoaderDecorator.load(CacheAwareLibraryLoaderDecorator.java:49)

@jingtang10 @pld @f-odhiambo @dubdabasoduba

[jingtang10]

@vitorpamplona @brynrhodes can you help here please?

[vitorpamplona]

Fixed on #1603