[DEMIS2SORMAS] Process multiple samples via one message [8]

[kwa20]

Problem Description

Currently, lab messages are split when multiple samples are referenced in one DEMIS notification. This is a problem for higher health institutions and the contextual meaning of the lifecycle management. Thus, we were asked to adjust the processing in SORMAS and not split a single notification depending on the number of referenced samples. This is also supposed to make processing easier for users.
The following two problems have to be addressed:

• Not splitting notifications into multiple messages, therefore allowing to process multiple samples
• Being prepared that reporters, especially laboratories, do not reference samples correctly in their notifications which therefore leads to a required interpretation of them being separate samples.

Proposed Change

• Only create a single lab message for notifications where multiple samples are referenced in a DEMIS notification.
• When processing, display an info pop-up informing the user that multiple samples are referenced in the message
  • Header: "Multiple samples"
  • Message: "This message references multiple samples. You will be guided through the creation of the number of referenced samples. Note that potential sample duplicates can be prevented by selecting the existing or previously selected sample in the respective 'pick or create sample' windows."
• When processing lab messages that contain multiple sample references, trigger the sample and pathogen creation forms consecutively depending on the number of referenced samples and prefill the respective samples depending on the message content. Also trigger the pick or create sample forms for each sample in order to allow the selection of potentially duplicate or existing samples.

Acceptance Criteria

Implementation Details

With the proposed change, there comes the problem that lab messages currently can just store information about one sample. With the changes proposed above, it is necessary to store data of several samples per lab message. Therefore, similar to test reports which enable data of several pathogen tests to be available per lab message, sample reports shall be introduced:

• create a sampleReport dto and entity class
• migrate all attributes starting with sample from the external message (current entity storing lab message data) to the sample report. Also do this for the attributes labSampleId and specimenCondition.
• the sample report shall also hold a reference to a lab message

This change affects the following existing structures and features:

Currently, each test report references a lab message. As a test report stores information about a pathogen test, it is necessary for them to have a reference to sample data. Right now, each test report therefore refers to a lab message (containing data of one sample). As, with the proposed change, there will be information about several samples per lab message, this reference has to be changed. Test reports have to refer to sample reports now. Thus, the new overall structure shall look like:

• make test reports refer to sample reports instead of lab messages
• make sure that the creation/deletion of a lab message leads to the creation/deletion of all associated sample reports and test reports.

Currently, at the end of processing a lab message, there is a relation stored: The lab message gets a reference to the sample asssociated through the processing. With this change, several samples can be associated. Therefore, several references need to be stored. That's why this reference shall be moved from the lab message to the sample report.

• additionally move the sample attribute from the external message to the sample report

In the sample list (when viewing a case or contact), there is the option to open associated lab messages

• For this to still work, the lookup of associated lab messages has to happen via sample reports now.

The changes described above also require adaptions in the lab message mapper (sample values can not be taken from the lab message anymore, but from the sample report) and database migration.
Furthermore, S2S-sharing of lab messages is affected, because sample reports now also have to be transferred there.

There exist shortcuts for processing lab messages (#5729). These shortcuts are currently designed to check for just one sample.
After the changes proposed here, they shall just be triggered when there is exactly one sample report for the lab message to be processed. One could also consider modifying the shortcuts to take into account multiple samples, but that is probably too complex to be of use.

Additional Information

Related SORMAS-DEMIS-adapter ticket: #63

[FredrikSchaefer]

I fear this will technically be quite complex, and require significant changes in SORMAS and SORMAS-DEMIS-adapter.
Also, this will increase the complexity of processing external messages.

For each sample, there may be popups triggered (e.g. for updating the disease variant of the related case, or for changing the overall test result of the sample). Those popups should immediately show when saving that one sample, right? So before triggering the pick or create sample form for the next sample...

One possibility might be to introduce SampleReports as entities. Similar to TestReports (which hold information about PathogenTests), SampleReports could hold information about Samples. Each TestReport would then refer to one SampleReport, and each SampleReport would refer to one ExternalMessage.

[FredrikSchaefer]

I had to improvise on some aspects, as they were not specified in the ticket.

• related messages are just looked for, when there is one sample report in the message to be processed. Multiple sample reports can lead to complex scenarios that are not yet thought of.
• corrections are also just made for the first sample report of a message
• I modified the update disease variant for a case popup logic, so that it just shows once per disease variant. Before, it popped up for every pathogen test with a disease variant other than the case's one. When processing multiple samples, that can be quite often.

[abrudanancuta]

The ticket will be reopen because of the appearance of the Update disease variant pop-up being hidden in the background of the process demis message.

Steps to reproduce:

1. Have prepared a message containing 2 Samples(1 Sample having 2 Pathogen tests, and the other Sample only 1 Pathogen test)
2. All 3 Pathogen test have as disease Covid and disease variant the Alpha code
3. Send the message from above to be fetched by Sormas
4. Process the new message from above
5. Choose to create a new Person and a new Case
6. At Case choose only Covid without selecting the variant
7. Continue by processing the first sample and then the second sample. Each sample will have the flow as new Sample
8. Choose to Save and open the Case
9. Observe the number of the appearance of the pop-up to update the disease variant

Actual results:
The pop-up appears 3 times because at each pathogen saved it was triggered in the background of the Process message window

Expected results:
The pop-up about update disease variant should appear after finishing the first Sample processing, after clicking YES(to update disease variant)it should update the case with the new variant. When finishing the 2nd Sample process the pop-up should not appear because the pathogen test has the same disease variant as the already variant saved on the Case.

[abrudanancuta]

Validated on local environment with the version: 1.76.0-SNAPSHOT (c76bbe1)