Add GDPR FHIR Operation Support - $erase

[prb112]

Background
As part of GDPR support, HL7 FHIR is discussing the $erase. The IBM FHIR Server should implement $erase.

Description

• Implement the $erase operation

Reference
FHIR R5 Roadmap
$erase

[lmsurpre]

HAPI FHIR Server implements a similar operation called $expunge (formerly $purge): https://smilecdr.com/docs/fhir_repository/deleting_data.html#the-expunge-operation

[kmbarton423]

need to consider storing compartment relationships outside of parameter tables

[punktilious]

Compartment relationships are now computed during ingestion and stored as internal token parameters. This information could be leveraged to drive a hard-delete for a patient. To do this properly we need to consider the volume of data likely to be associated with a patient and whether or not it would be possible to delete all that data in a single transaction (taking into account the constraints of log space and transaction time). An asynchronous batch process may be needed so that the deletes can be spread over multiple transactions.

[lmsurpre]

The really tricky thing here is that we only store parameters for the latest version of a given resource.

Problem scenarios:

Soft-deleted resources

1. version 1 of a resource points to Patient/abc
2. user soft-deletes this resource (which essentially becomes version 2 of this resource)

Now we have a request to delete all resources for Patient/abc ...good luck finding the historical version of this resource

Changing ownership

1. version 1 of a resource points to Patient/123oops
2. version 2 of the resource points to Patient/123fixed

Now we have a request to delete all resources for Patient/123oops; should this resource which now belongs to Patinet/123fixed be hard deleted?

Lee's proposal

Its scenarios like this that can be very tricky. Unless we want to start storing compartment membership for each version of each resource, I think it might be better to 'punt' on the "erase everything about patient XYZ" and instead implement a single-resource hard-delete; then let the client decide which resources to purge.

How this might play out in a real solution:
Using our fhir-notification mechanism, there is some other component listening for resource creates and updates and associating ResourceType+ResourceId+Version with a given patient. This is that component's sole responsibility.

Then, when it comes time to delete all resources associated with that patient, this component does a simple lookup in its map to decide which resources should get blown away and drives the erasure of those resources. Note: the "Changing ownership" scenario makes this especially tricky and so it might make sense to either disable that via a PersistenceInterceptor or otherwise have policy in place for either mitigating or managing this scenario.

[prb112]

I've started working on this at a design level and considered Lee and Robin's comments. I'll have a Dev Team meeting on Wednesday to discuss more on the design and implementation.

[JohnTimm]

Ran through the Postman collection that @prb112 provided to verify the feature.