Skip to content

v4.1.0

Choose a tag to compare

@isc-tdyar isc-tdyar released this 28 Sep 16:04
· 502 commits to main since this release
477f9c3

An ISC FHIR repository can now be projected as a named graph and read patient by
patient. Concepts in a terminology graph can be walked to the resources that
carry their codes and ranked with PPR. This release also passes the openCypher
TCK: 3896/3896 eligible scenarios, strict and typed. It fixes two graph-scoping
bugs from 4.0.0 (PPR walked every graph, delete_nodes erased an id from every
graph), and 4.0.0's migration now completes on 3.x installs.

Install

pip install --upgrade iris-vector-graph==4.1.0

Upgrading from 4.0.0

  1. Redeploy the Graph.KG.* classes. FHIRGraph, FHIRGraphSyncTask and
    GraphMode are new; PageRank, EdgeScan and Eraser changed.
  2. Run initialize_schema(). It adds rdf_edges.ekey (every existing row
    becomes ekey = 0; ^KG and the ledger are untouched) and code_crosswalk,
    in resumable steps.
  3. Only namespaces with a FHIR repository get FHIR tables. initialize_schema
    does not create fhir_graphs, fhir_unresolved, fhir_definitions or
    fhir_canonical_refs. The first fhir_graph_register creates them, and only
    when HS_FHIRServer.Repo exists. GraphSchema.get_fhir_graph_schema_sql()
    returns their DDL.

Behaviour changes to check:

  • store.delete_nodes and store.delete_edges now affect only the default
    graph unless given graph=. In 4.0.0 they affected every graph.
  • New boolean writes store 'true' / 'false' instead of '1' / '0'. Reads
    and equality still match the old spellings, and existing rows are not
    rewritten.
  • properties(n) returns a Cypher map ({level: 9001}), not a key/value list.
  • kg_PERSONALIZED_PAGERANK now walks only the graph it is given. In 4.0.0 it
    walked every graph.

If you ran a pre-release build from main with FHIR graphs, its first sync
after the upgrade re-derives every resource once. fhir_graphs.interp_version
is NULL on those graphs, and on a large graph that sync takes about as long as a
rebuild. Run fhir_reinterpret(graph) first to pick the time.

FHIR repository as a graph

  • fhir_graph_register() registers the namespace's HS.FHIRServer repository
    (R4 JsonAdvSQL) as graph fhir:<NAMESPACE>:<package>. A node is a resource
    and an edge is an indexed reference. Codes and properties stay in the
    repository and are read live.
  • fhir_graph_rebuild reconciles by diff and keeps vectors on surviving nodes.
    fhir_graph_sync applies changes in batches of 200, one transaction per
    batch. fhir_graph_schedule runs it from the Task Manager.
  • Canonical references are edges now. So are links read from the resource body
    (Type.element, extension:<url>). fhir_link_report explains every
    unresolved one.
  • CLI: ivg fhir register|rebuild|sync --once|status|schedule|links.

Patient-centred interpretation (base R4, no profiles)

  • New derived edge in_patient_compartment (resource → Patient), taken from the
    R4 patient-compartment params. Its via qualifier lists those params.
  • Every node carries category and meta_profile. meta_profile is what the
    writer claimed; nothing validates it. HL7 and US Core category codes also
    become labels (Laboratory, VitalSigns).
  • params="clinical" resolves concepts through code, value-concept,
    component-code, component-value-concept and Medication code. A resolved
    Medication brings its MedicationRequest, MedicationStatement,
    MedicationAdministration and MedicationDispense resources.
  • fhir_concept_ppr(group_by="patient") ranks patients by summing their
    compartment resources' scores, and returns the top contributors.
  • The default PPR exclusions (FHIR_PPR_EXCLUDE) are in_patient_compartment,
    Provenance.target and Provenance.entity, so Provenance hubs do not soak up
    mass.
  • /api/cypher with fhir_patient_id takes its anchors from the synced graph
    and falls back to the external bridge. The response reports which one was used
    (anchor_source).
  • Reports, computed live and never stored: fhir_coverage_report(graph) and
    fhir_concept_gaps(graph, params=...).

Contract:
docs/FHIR_GRAPH.md#interpretation-contract.

From concepts to ranked resources

  • code_crosswalk maps a (system, code) pair to a concept node, with relation
    exact, broader, narrower or related.
  • fhir_expand_concepts, fhir_resolve_concepts and fhir_concept_ppr chain
    concept expansion, code resolution and PPR. Hierarchy direction is
    direction="in"|"out"|"both".
  • Genomics: measured on 377 HL7 Genomics Reporting IG examples. Genes sit in
    Observation components, so they need component-value-concept or
    params="clinical".

openCypher TCK: 3896/3896, strict and typed

  • Strict mode is the harness default. Side effects are checked by snapshot diff,
    and error scenarios must raise the expected error type.
  • Typed scoring: a cell passes only if its value and its Cypher type both match.
    Fixes for relationship properties, map field access, mixed-type lists and
    WITH aliases moved typed scoring from 3859 to 3896.
  • 7 scenarios need parallel edges and pass only with multigraph mode on (see
    below).

Opt-in multigraph mode

engine.set_multigraph(graph, True) lets one graph hold parallel edges with the
same (source, predicate, target). It is off by default. While off, the
generated SQL and the ^KG layout are unchanged.

Fixed

  • kg_PERSONALIZED_PAGERANK walked every graph. It is now scoped to the graph
    it is given, in ObjectScript and in the Python fallback.
  • materialize_inference rules read across graphs, and inferred edges never
    reached ^KG, so BFS, PPR and concept expansion could not see them.
  • Arno default-graph PPR answered from a stale, pre-214 layout.
  • SSRF: the FHIR proxy in cypher_api.py fetched any base URL. It now accepts
    only FHIR_BASE_URL and IVG_FHIR_ALLOWED_BASES.
  • unified_clinical_pipeline never ranked anything: a swallowed TypeError
    looked like "no connectivity".
  • 4.0.0 migration on 3.x installs:
    • the rdf_edges rescue now resumes;
    • duplicate edges, labels and props are collapsed or quarantined;
    • legacy PRIMARY KEY (node_id) constraints are dropped;
    • delete_nodes / delete_edges are chunked and graph-scoped.
  • Cypher DELETE n with no USE GRAPH now deletes only the (graph, node) rows its
    MATCH bound. It used to delete labels and properties for that ID in every graph.
    Under USE GRAPH, a non-DETACH delete is no longer refused because of another
    graph's edges.
  • vector_search and edge_vector_search refused any score_threshold
    (SQLCODE -25, HAVING after ORDER BY). The threshold is a bound WHERE
    term now.
  • list_active_queries and GET /admin/queries never returned a row. They read
    INFORMATION_SCHEMA.CURRENT_STATEMENTS now; the row keys are unchanged.
  • Snapshots now export and restore fhir_unresolved, fhir_definitions and
    fhir_canonical_refs.
  • BulkLoader and migrate_to_nodepk lost the real SQLCODE (<LIST ERROR>)
    after another connection ran %BuildIndices.
  • Demo server: /bio failed on every search (it queried a table and port that no
    longer exist); it now reads the seeded graph bio:demo. /fraud crashed on first
    score (HTTP/2 without h2). Unverified figures on both pages are gone, and the
    landing page shows each backend's live status.

Deprecated, removed in 5.0

  • fhir_bridge.get_kg_anchors and unified_clinical_pipeline. Use
    fhir_resolve_concepts and fhir_concept_ppr.
  • POST /fhir-event.

Measured

FHIR medians in milliseconds on irishealth:2026.3.0AI.113.0. "Fixture" is 10
Synthea patients; "scale" is about 100. The baseline is the code before
interpretation.

Set Measure Baseline 4.1.0
fixture sync 35885 37348
fixture PPR 481 574
fixture grouped – 617
fixture coverage – 187
fixture gaps – 377
scale sync 47057 52401
scale PPR 8132 8654
scale grouped – 9024
scale coverage – 831
scale gaps – 1450

Known gaps

  • Snapshots do not carry fhir_graphs or code_crosswalk. After a restore,
    register the repository again, run fhir_graph_rebuild and re-add the
    crosswalk rows.
  • engine.delete_node(id) still deletes by ID alone, from every graph that holds
    it. A Cypher DELETE n with no USE GRAPH deletes the node in every graph its
    MATCH binds; say USE GRAPH to delete from one.
  • ^ArnoKG("KG","nkg_adj") is an all-graph snapshot. While it exists, Arno
    *_global calls answer across graphs.
  • Profiles are not interpreted or validated, and there are no non-Patient
    compartments and no Patient.link identity resolution.

A graph ID is not an authorisation boundary

The FHIR graph holds resource keys and references, which identify patients.
Deploy IVG in a FHIR namespace only for users who may already read the
repository's tables. For tenant isolation, use one IRIS namespace per tenant.


Full notes in
CHANGELOG.md.