HAPI Bundles and Export

HAPI Bundles and Export

The two main interactions that mimic-fhir has with HAPI is bundling and exporting. The bundle step sends resources to HAPI and validates on the way. Then to get the resources out the other end, the resources are exported from HAPI.

1. Bundling

Bundling resources helps to resolve referential integrity and accelerate the passing of resources to HAPI.

Bundles solve referential integrity

For mimic-fhir the referential integrity is important for these resources:

• Microbiology
  • ObservationMicroTest
  • ObservationMicroOrg
  • ObservationMicroSusc
• Medication
  • MedicaitonRequest
  • MedicationDispense
  • MedicationAdministration

Microbiology and medication resources have parent-child relationships that would fail if resources were passed individually.

MIMIC-FHIR Bundle Groupings

Resources from mimic-fhir needed to be organized to pass efficiently to HAPI. The bundles generated were:

Bundle	Resources	Notes
patient	Patient, Encounter, Condition, Procedure
spec	Specimen	The specimen are sent separately from micro and labs to simplify referencing
micro	ObservationMicroTest, ObservationMicroOrg, ObservationMicroSusc
lab	ObservationLabs
med	MedicationRequest, MedicaitonDispense, MedicationAdministration	Med bundle not currently working, medicaiton PR still in the works
icu_base	EncounterICU, ProcedureICU, MedicationAdministrationICU	May need to separate out medadmin since it is very large and no direct relations
icu_obs	ObservationChartevents, ObservationDatetimeevents, ObservationOutputevents	Chartevents may be separated out since they are huge and can work to optimize them sent alone

How to bundle in py_mimic_fhir

In the py_mimic_fhir package there are 2 bundle classes that are used to assist in bundling.

1. Bundle

• The bundle class creates a transaction bundle
• Lists of resources are passed to the Bundle to be added as entry's
• Once a bundle is ready, the Bundle.request function can be used to send the bundle to HAPI
  • If the bundle is invalid or there are errors in the resources, as OperationOutcome is returned with the issues
  • If the bundle is valid and no errors for its resources, the valid Bundle is returned

2. Bundler

• The bundler class is used to make mimic specific bundles
• A patient_id is passed into the bundler
• All resources associated with that patient are then created into the bundle groupings specified above
• Once all bundles are generated for a patient, they can be sent individually or all together to HAPI

To handle any errors in the bundles, an ErrBundle class was created. The ErrBundle class reorganizes the failed bundle, sending the issue and fhir ids to a bundle error log. These failed bundles can be rerun from this error log once the source of error has been fixed using the rerun_bundle_from_file function.

2. Exporting

To export resources from HAPI the bulk export feature is used. The $export keyword is used to trigger the export. The steps for export are:

• POST export request
  • Example request: http://localhost:8080/fhir/$export?_type&?_typeFilter=?_profile=<profile_url>
    • type: the resource type
    • typeFilter: the resource type followed by filter paramaters
    • _profile: One of may filter paramaters. Specifies the profile the resource is set to
  • Response from the server will include a polling location to get the export download location
• GET exported resource download location
  • Send GET request to polling location received in the initial POST request
  • The request to the polling location will return either the resources download location or a message saying not ready yet
  • Keep polling until resources are ready
• GET exported resources
  • Using the download location URL get the resources
  • The resources will come in a binary format (base64)
• Write resource to NDJSON
  • Decode resources since they come in a binary format from the server
  • Write all resources of the same type to the same NDJSON