Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrates Workflow to CQL Evaluator 3.0.0-PRE9-SNAPSHOT #2259

Merged
merged 14 commits into from
Oct 11, 2023

Conversation

vitorpamplona
Copy link
Collaborator

@vitorpamplona vitorpamplona commented Oct 11, 2023

Migrates the workflow, knowledge, workflow-testing, and workflow/benchmark to the new Clinical Reasoning API.

This PR should cover most if not all of the current uses of the workflow module with updated test cases.

Done:

  • Adjusted dependencies to the new version. Deleted most of the old hacks that were necessary to work (caffeine).
  • Migrated FhirDal implementations to Repostiory implementations
  • Deleted Content, Terminology, and Retrieve Providers; and merges some of their code into Repository.
  • Migrated Test cases to the new testing and repository structure.
  • Created a new IGInputStream Repository using Android resources to simulate the IGFile Repository of test cases from upstream.
  • Updates all test cases that were originally from upstream to make them run on AndroidJunit

TODO (next PRs):

  • Implement FHIR Operators (lookup/expand/in) that were in the outdated Terminology Provider
  • Fully implement the Repository interface's Search function with nested AND/OR conditionals.
  • Better merge the Engine's Search interface with the new Search interface from HAPI (search param interface vs client search param interface)
  • Update the ProxyRepository to use FhirEngine and Knowledge repository for terminology (split by type instead of trying one after another)
  • Make sure the new Repository classes cover all of the outdated Retrieve Provider implementations
  • Review OpioidRec10PatientView test case
  • Delete the list(null) hack in the measureEvaluate function when this is fixed: Population Measure crashes when subjectIds is null or empty cqframework/clinical-reasoning#358
  • Create a Gradle Task to automatically generate the contents.txt file in every test repository being used
  • Fix Jackson's multi-version dependency scheme due to the need to support minSDK 24 and newer versions of Jackson at the same time
  • Merge the UcumService property of SimpleWorkerContext and the correct caching of StructureDefinitions from HapiWorkerContext to replace today's DualHapiWorkerContext
  • Expose just one Ucum service from Commons -> UnitConverter.ucumService for all engine and workflow needs
  • Fix an upstream bug that incorrectly returns 2022-12-31T00:00:00 instead of 2021-12-31T23:59:59 in some measure evaluates

Type
Feature

Screenshots (if applicable)

Checklist

  • I have read and acknowledged the Code of conduct.
  • I have read the Contributing page.
  • I have signed the Google Individual CLA, or I am covered by my company's Corporate CLA.
  • I have discussed my proposed solution with code owners in the linked issue(s) and we have agreed upon the general approach.
  • I have run ./gradlew spotlessApply and ./gradlew spotlessCheck to check my code follows the style guide of this project.
  • I have run ./gradlew check and ./gradlew connectedCheck to test my changes locally.
  • I have built and run the demo app(s) to verify my change fixes the issue and/or does not break the demo app(s).

@omarismail94
Copy link
Contributor

@vitorpamplona , there are two Instrumentation tests that are failing:

  • com.google.android.fhir.workflow.CqlBuilderAndroidTest.shouldCompileAndAssembleImmunityCheck
  • com.google.android.fhir.workflow.PlanDefinitionProcessorAndroidTest.testANCDT17

the errors are due to the expected and the actual values being the same, but not in order.

I've figure out the code to resolve this. In CQLBuilder, change line 200 to:

      JSONAssert.assertEquals(expectedElmJsonAsset, translator.toJson(), false)

and in XMLAssert change line 30 to:

        .withNodeMatcher(DefaultNodeMatcher(ElementSelectors.byNameAndAllAttributes))

can you make those changes on your end please?

@omarismail94
Copy link
Contributor

@vitorpamplona my bad, testANCDT17 is not resolved by what I suggested. It fails locally as well when I run the test on my device

Copy link
Collaborator

@jingtang10 jingtang10 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what a monumental effort! thanks @vitorpamplona!

@jingtang10 jingtang10 merged commit 633b482 into google:master Oct 11, 2023
5 checks passed
santosh-pingle pushed a commit to santosh-pingle/android-fhir that referenced this pull request Oct 18, 2023
* Updates dependencies

* Updates test cases

* Removes moxy and antlr from dependencies.

* Migrates the workflow to the new API

* Migrates workflow testing to the new API

* Fixes workflow benchmark module

* Fixes workflow benchmark module

* Adds contents.txt file to all test repos so it can be loaded on androidTest

* Normalizes path before trying to open files

* Uses contents.txt when running tests on Android

* Spotless

* Updating Json and XML comparisons to pass when elements are just out of order.

* Ignores testANCDT17 for now
jingtang10 added a commit that referenced this pull request Oct 19, 2023
* Update material3 theme.

* Support m3 night mode.

* Address review comments.

* Address review comments.

* Apply m3 default style to survey button.

* Use default typography.

* Fix broken dependabot.yaml syntax (fixes #2195) (#2234)

* Moves commons to Hapi 6.8, removes caffeine and forces guava-android (#2241)

* Bump up common library version to 0.1.0-alpha05 (#2243)

* Update material3 theme in demo app. (#2211)

* Update material3 theme.

* Support m3 night mode.

* Address review comments.

* Address review comments.

* Change the color order.

---------

Co-authored-by: Santosh Pingle <spingle@google.com>

* Fix broken dependabot.yaml syntax again (re. #2195) (#2242)

* SdkSuppress FhirSyncWorkerBenchmark oneTimeSync_50patients (re. #2235) (#2236)

* Allow custom package server in Knowledge Manager (#2244)

* Allow custom package server in Knowledge Manager

* Fix downloader test

* Fix workflow library

* Update knowledge/src/main/java/com/google/android/fhir/knowledge/files/NpmFileManager.kt

Co-authored-by: aditya-07 <aditya.pu2010@gmail.com>

* Fix build

* Add try catch for package download

---------

Co-authored-by: aditya-07 <aditya.pu2010@gmail.com>

* Bump up the version of knowledge manager (#2258)

* Migrates engine to HAPI 6.8 while forcing Workflow to stay on 6.0 (#2245)

* Migrates engine to HAPI 6.8 while forcing Workflow to stay on 6.0

* comment

* Migrates Workflow to CQL Evaluator 3.0.0-PRE9-SNAPSHOT (#2259)

* Updates dependencies

* Updates test cases

* Removes moxy and antlr from dependencies.

* Migrates the workflow to the new API

* Migrates workflow testing to the new API

* Fixes workflow benchmark module

* Fixes workflow benchmark module

* Adds contents.txt file to all test repos so it can be loaded on androidTest

* Normalizes path before trying to open files

* Uses contents.txt when running tests on Android

* Spotless

* Updating Json and XML comparisons to pass when elements are just out of order.

* Ignores testANCDT17 for now

* LaunchContext for initialExpression (#2025)

* Use launchContext for initialExpression

* Test ResourceMapper

* Fix test

* Fix launchContexts for demo app when editing patient

* spotlessApply

* WIP

* Revert "WIP"

This reverts commit d2d53a2.

* spotlessApply

* Fix test

* Refactor validateLaunchContextExtension

* Remove QuestionnaireLaunchContextSet enum class

* Rename vars and functions

* Add code comment for MoreResourceTypes.kt

* Unit testing

* Fix post-merge-conflict

* spotlessApply

* Address review

* spotlessApply

* Revert un-intended changes

* Fix failing checks

* Remove check of must contain 2 sub-extensions

* Update Kdoc

* Revert

* validate launch context when using populate public API

* Renames patientId param to subject to: (#2263)

1. clarify the need to send a full reference "type/id" instead of just the ID
2. match the evaluator's name

* Follow up PR of 2025 (#2265)

* Use launchContext for initialExpression

* Test ResourceMapper

* Fix test

* Fix launchContexts for demo app when editing patient

* spotlessApply

* WIP

* Revert "WIP"

This reverts commit d2d53a2.

* spotlessApply

* Fix test

* Refactor validateLaunchContextExtension

* Remove QuestionnaireLaunchContextSet enum class

* Rename vars and functions

* Add code comment for MoreResourceTypes.kt

* Unit testing

* Fix post-merge-conflict

* spotlessApply

* Address review

* spotlessApply

* Revert un-intended changes

* Fix failing checks

* Remove check of must contain 2 sub-extensions

* Update Kdoc

* Revert

* validate launch context when using populate public API

* Address Jing's Review

- #2025 (comment)
- #2025 (comment)

* Bump com.squareup:kotlinpoet from 1.12.0 to 1.14.2 (#2255)

Bumps [com.squareup:kotlinpoet](https://github.com/square/kotlinpoet) from 1.12.0 to 1.14.2.
- [Release notes](https://github.com/square/kotlinpoet/releases)
- [Changelog](https://github.com/square/kotlinpoet/blob/main/docs/changelog.md)
- [Commits](square/kotlinpoet@1.12.0...1.14.2)

---
updated-dependencies:
- dependency-name: com.squareup:kotlinpoet
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jing Tang <jingtang@google.com>

* Bump androidx.build.gradle.gcpbuildcache (#2266)

Bumps androidx.build.gradle.gcpbuildcache from 1.0.0-beta01 to 1.0.0-beta05.

---
updated-dependencies:
- dependency-name: androidx.build.gradle.gcpbuildcache
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Omar Ismail <44980219+omarismail94@users.noreply.github.com>

* Bump com.spotify.ruler:ruler-gradle-plugin from 1.2.1 to 1.4.0 (#2253)

Bumps [com.spotify.ruler:ruler-gradle-plugin](https://github.com/spotify/ruler) from 1.2.1 to 1.4.0.
- [Release notes](https://github.com/spotify/ruler/releases)
- [Changelog](https://github.com/spotify/ruler/blob/main/changelog.md)
- [Commits](spotify/ruler@v1.2.1...v1.4.0)

---
updated-dependencies:
- dependency-name: com.spotify.ruler:ruler-gradle-plugin
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump com.diffplug.spotless:spotless-plugin-gradle from 6.21.0 to 6.22.0 (#2252)

Bumps [com.diffplug.spotless:spotless-plugin-gradle](https://github.com/diffplug/spotless) from 6.21.0 to 6.22.0.
- [Changelog](https://github.com/diffplug/spotless/blob/main/CHANGES.md)
- [Commits](diffplug/spotless@gradle/6.21.0...gradle/6.22.0)

---
updated-dependencies:
- dependency-name: com.diffplug.spotless:spotless-plugin-gradle
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump actions/checkout from 3 to 4 (#2248)

Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](actions/checkout@v3...v4)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jing Tang <jingtang@google.com>

* Update dependencies in engine colab (#2268)

* Update dependencies in engine colab

* update datacapture

* Bump styfle/cancel-workflow-action from 0.7.0 to 0.12.0 (#2250)

Bumps [styfle/cancel-workflow-action](https://github.com/styfle/cancel-workflow-action) from 0.7.0 to 0.12.0.
- [Release notes](https://github.com/styfle/cancel-workflow-action/releases)
- [Commits](styfle/cancel-workflow-action@0.7.0...0.12.0)

---
updated-dependencies:
- dependency-name: styfle/cancel-workflow-action
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump actions/upload-artifact from 2 to 3 (#2249)

Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 2 to 3.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](actions/upload-artifact@v2...v3)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Update the version of resource after updates are downloaded from the server (#2272)

* Update the version of resource after updates are downloaded from the server

* Review changes: Added tests and refactored code

* Review comments updates

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: Santosh Pingle <spingle@google.com>
Co-authored-by: Michael Vorburger ⛑️ <vorburger@google.com>
Co-authored-by: Vitor Pamplona <vitor@vitorpamplona.com>
Co-authored-by: Jing Tang <jingtang@google.com>
Co-authored-by: aditya-07 <aditya.pu2010@gmail.com>
Co-authored-by: aditya-07 <adityakhajuria@google.com>
Co-authored-by: Fikri Milano <fikrimilano1@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Omar Ismail <44980219+omarismail94@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Complete
Development

Successfully merging this pull request may close these issues.

None yet

4 participants