diagnosis was not extracted and make conversion to CONDITION_OCCURRENCE table

[xinyuejohn]

Hi, I just noticed that diagnosis was was not extracted and transformed to CONDITION_OCCURRENCE table. Since diagnosis are very important in many pipelines, do you have plans to also support this?

Thanks!

[USM-CHU-FGuyon]

Hi, we do not expect to support this in the close future. We agree that it would be a great addition, but did not yet find the time to do this. However, I would be happy to review pull requests for supporting diagnoses in any of the included databases.

[xinyuejohn]

@USM-CHU-FGuyon Hi, I'm interested in incorporating diagnosis data from MIMIC-IV into the current pipeline. Before I begin, I would appreciate any guidance or implementation plan you could offer to start this addition. Thanks!

[USM-CHU-FGuyon]

Hi, Thank you for your help ! I haven't had time to get on this in the past months...
I can prepare the base classes and files and come up with some guidance on the steps I see.

Getting back to you next Tuesday with this !

[USM-CHU-FGuyon]

Hi,

I pushed some changes : 7156be1 in which I added diagnosis processing through {dataset}_DiagProcessor classes.

Two diagnoses were extracted as a minimal working example.

The harmonized diagnoses are found there auxillary_files/diagnoses.json.

What's next

If you are still interested in adding more diagnoses into the pipeline, what needs to be done is to create a more comprehensive diagnoses.json file. This is the class where it happens: omop_diagnoses

This file should contain the mapping from ICD codes to OMOP concept_ids. Ideally, we only really need to find a conversion tool from ICD to OMOP to build the diagnoses.json file.

In the Influenza example, I simplified the original ICD codes by mapping all Influenza-related codes to "Influenza". I think this is acceptable as the "condition_occurrence_source_value" stays in the OMOP file, but it would be generally better to find a 1-to-1 correspondance between ICD and OMOP.

This file should be created at step 0_prepare_files, just like medications.json.
I already added the DiagnosesMapping class.

Notes

1. For now, condition occurrence still has some duplicate rows/condition_occurrence_id, this will be fixed in the next commit
2. Procedures can and will likely be handled in a similar manner.

Questions
I am available if you have any question or need any help.

Thanks for your interest in this work,
Matthieu