Medical-Diff-VQA (originally MIMIC-Diff-VQA) is a large-scale dataset for visual question answering in medical chest x-ray images. This repository provides the code for generating Medical-Diff-VQA dataset, which is proposed in our paper "Expert Knowledge-Aware Image Difference Graph Representation Learning for Difference-Aware Medical Visual Question Answering"
For more information about the dataset and the method, please refer to our paper or project page.
For the code of the method, please refer to EKAID.
The Medical-Diff-VQA dataset is now available on Physionet.
To generate a new Medical-Diff-VQA dataset(due to the randomness, the generated dataset will not be 100% the same as our provided one), please follow the steps below:
-
Enter the 'code' directory
cd code -
Prepare for the mimic_all.csv (MIMIC-CXR-JPG needs to be ready)
python get_mimic_all.py -p <path_to_mimic_cxr_jpg>
-
Extract the intermediate KeyInfo json dataset. The <path_to_reports_folder> refers to the "files" folder that is unzipped from the mimic-cxr-reports.zip file in the MIMIC-CXR database.
python question_gen.py -j -r <path_to_reports_folder>
-
Generate the full version of question answer pairs
python question_gen.py -q
Alternatively, you can execute step 3 and step 4 simultaneously by running:
python question_gen.py -j -q