Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Load UMLS data into TermGraph and allow triggering in cloud container #2795

Merged
merged 7 commits into from
Sep 24, 2021

Conversation

ibmatc
Copy link
Contributor

@ibmatc ibmatc commented Sep 22, 2021

These changes enable loading UMLS data (Concepts, Relationships, and now map edges between Snomed and ICD) into FHIR TermGraph. This also includes example yml for setting up Cassandra and ElasticSearch for JanusGraph to use.

Signed-off-by: Adam T. Clark atclark@us.ibm.com

loadTermGraph.sh Outdated Show resolved Hide resolved
@prb112
Copy link
Contributor

prb112 commented Sep 22, 2021

 on: pull_request
Integration on: pull_request
3
Build and Tests on: pull_request
Integration - Audit on: pull_request
3
e2e-audit (openjdk11, kafka)
Integration - Windows on: pull_request
2
Integration - Persistence on: pull_request
3
e2e-audit (openjdk11, kafka)
failed 8 minutes ago in 2m 30s
3s
20s
6s
2m 1s
[INFO] fhir-audit ......................................... SUCCESS [  0.899 s]
[INFO] fhir-registry ...................................... SUCCESS [  5.265 s]
[INFO] fhir-term .......................................... SUCCESS [  0.721 s]
[INFO] fhir-term-graph .................................... FAILURE [  6.405 s]
[INFO] fhir-provider ...................................... SKIPPED
[INFO] fhir-term-remote ................................... SKIPPED
[INFO] fhir-profile ....................................... SKIPPED
[INFO] fhir-path .......................................... SKIPPED
[INFO] fhir-validation .................................... SKIPPED
[INFO] fhir-ig-us-core .................................... SKIPPED
[INFO] fhir-ig-carin-bb ................................... SKIPPED
[INFO] fhir-ig-mcode ...................................... SKIPPED
[INFO] fhir-ig-davinci-pdex-plan-net ...................... SKIPPED
[INFO] fhir-ig-davinci-pdex-formulary ..................... SKIPPED
[INFO] fhir-ig-davinci-pdex ............................... SKIPPED
[INFO] fhir-ig-davinci-hrex ............................... SKIPPED
[INFO] fhir-ig-spl ........................................ SKIPPED
[INFO] fhir-search ........................................ SKIPPED
[INFO] fhir-persistence ................................... SKIPPED
[INFO] fhir-persistence-schema ............................ SKIPPED
[INFO] fhir-persistence-jdbc .............................. SKIPPED
[INFO] fhir-notification .................................. SKIPPED
[INFO] fhir-notification-websocket ........................ SKIPPED
[INFO] fhir-notification-kafka ............................ SKIPPED
[INFO] fhir-notification-nats ............................. SKIPPED
[INFO] fhir-server ........................................ SKIPPED
[INFO] fhir-operation-test ................................ SKIPPED
[INFO] fhir-operation-validate ............................ SKIPPED
[INFO] fhir-operation-document ............................ SKIPPED
[INFO] fhir-operation-healthcheck ......................... SKIPPED
[INFO] fhir-operation-apply ............................... SKIPPED
[INFO] fhir-operation-convert ............................. SKIPPED
[INFO] fhir-operation-term ................................ SKIPPED
[INFO] fhir-operation-term-cache .......................... SKIPPED
[INFO] fhir-operation-reindex ............................. SKIPPED
[INFO] fhir-operation-bulkdata ............................ SKIPPED
[INFO] fhir-operation-everything .......................... SKIPPED
[INFO] fhir-operation-erase ............................... SKIPPED
[INFO] fhir-bulkdata-webapp ............................... SKIPPED
[INFO] fhir-client ........................................ SKIPPED
[INFO] fhir-server-webapp ................................. SKIPPED
[INFO] fhir-server-test ................................... SKIPPED
[INFO] fhir-smart ......................................... SKIPPED
[INFO] fhir-bucket ........................................ SKIPPED
[INFO] fhir-swagger-generator ............................. SKIPPED
[INFO] fhir-openapi ....................................... SKIPPED
[INFO] fhir-install ....................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  55.365 s
[INFO] Finished at: 2021-09-22T19:06:21Z
[INFO] ------------------------------------------------------------------------
Error:  Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.1:compile (default-compile) on project fhir-term-graph: Compilation failure: Compilation failure: 
Error:  /home/runner/work/FHIR/FHIR/fhir-term-graph/src/main/java/com/ibm/fhir/term/graph/loader/impl/UMLSTermGraphLoader.java:[84,9] no suitable constructor found for AbstractTermGraphLoader(java.util.Map<java.lang.String,java.lang.String>,org.apache.commons.configuration.Configuration)
Error:      constructor com.ibm.fhir.term.graph.loader.impl.AbstractTermGraphLoader.AbstractTermGraphLoader(java.util.Map<java.lang.String,java.lang.String>,org.apache.commons.configuration2.Configuration) is not applicable
Error:        (argument mismatch; org.apache.commons.configuration.Configuration cannot be converted to org.apache.commons.configuration2.Configuration)
Error:      constructor com.ibm.fhir.term.graph.loader.impl.AbstractTermGraphLoader.AbstractTermGraphLoader(java.util.Map<java.lang.String,java.lang.String>,com.ibm.fhir.term.graph.FHIRTermGraph) is not applicable
Error:        (argument mismatch; org.apache.commons.configuration.Configuration cannot be converted to com.ibm.fhir.term.graph.FHIRTermGraph)
Error:  /home/runner/work/FHIR/FHIR/fhir-term-graph/src/main/java/com/ibm/fhir/term/graph/loader/impl/SnoMedICD10MapTermGraphLoader.java:[154,9] no suitable constructor found for AbstractTermGraphLoader(java.util.Map<java.lang.String,java.lang.String>,org.apache.commons.configuration.Configuration)
Error:      constructor com.ibm.fhir.term.graph.loader.impl.AbstractTermGraphLoader.AbstractTermGraphLoader(java.util.Map<java.lang.String,java.lang.String>,org.apache.commons.configuration2.Configuration) is not applicable
Error:        (argument mismatch; org.apache.commons.configuration.Configuration cannot be converted to org.apache.commons.configuration2.Configuration)
Error:      constructor com.ibm.fhir.term.graph.loader.impl.AbstractTermGraphLoader.AbstractTermGraphLoader(java.util.Map<java.lang.String,java.lang.String>,com.ibm.fhir.term.graph.FHIRTermGraph) is not applicable
Error:        (argument mismatch; org.apache.commons.configuration.Configuration cannot be converted to com.ibm.fhir.term.graph.FHIRTermGraph)

Compilation failures

@JohnTimm
Copy link
Collaborator

Good stuff!

Signed-off-by: Adam T. Clark <atclark@us.ibm.com>
Adam T. Clark added 3 commits September 23, 2021 14:54
Signed-off-by: Adam T. Clark <atclark@us.ibm.com>
Signed-off-by: Adam T. Clark <atclark@us.ibm.com>
Signed-off-by: Adam T. Clark <atclark@us.ibm.com>
@ibmatc ibmatc force-pushed the atc-trigger-umls-term-graph-loader branch from c277b6a to 0a1dc81 Compare September 23, 2021 21:08
Signed-off-by: Adam T. Clark <atclark@us.ibm.com>
Copy link
Contributor

@prb112 prb112 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Signed-off-by: Adam T. Clark <atclark@us.ibm.com>
Comment on lines +104 to +107
<dependency>
<groupId>com.ibm.cos</groupId>
<artifactId>ibm-cos-java-sdk</artifactId>
</dependency>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We were already using this in fhir-bulkdata-webapp, but this is a new dependency for the fhir-server-webapp. I think it pulls in much of the aws sdk.
Since its not actually needed by the server during runtime, I think we should work to split out this new loading tool work so that it doesn't end up in the webapp.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agreed #2799

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, I read that as more of creating a utility class to encapsulate the code that accesses COS. i didn't think it went quite as far as what i'm suggesting here which is to split fhir-term-graph into two totally separate modules:

  1. the loading stuff the creates/populates the graph db
  2. the provider stuff that uses the graph and ships with the server

Signed-off-by: Adam T. Clark <atclark@us.ibm.com>
Copy link
Member

@lmsurpre lmsurpre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ibmatc !

@lmsurpre lmsurpre merged commit b3e3494 into main Sep 24, 2021
@lmsurpre lmsurpre deleted the atc-trigger-umls-term-graph-loader branch September 24, 2021 19:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants