MaCo

The code of 'Enhancing Representation in Radiography-Reports Foundation Model: A Granular Alignment Algorithm Using Masked Contrastive Learning' Some code is borrowed from MAE, huggingface, and MRM.

Environmental preparation

conda create -n MaCo python=3.8
conda activate MaCo
pip install -r requirements.txt

Links to download datasets

• MIMIC-CXR-JPG

• NIH ChestX-ray

• CheXpert

• RSNA Pneumonia

• SIIM-ACR Pneumothorax

• MS-CXR

Datasets splits

In the directory DatasetsSplits, we provide dataset splits that may be helpful for organizing the datasets.

We give the train/valid/test splits of CheXpert, NIH ChestX-ray, RSNA Pneumonia, and SIIM-ACR Pneumothorax.

Pretraining

Adjust the necessary paths and perform the following code:

CUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --nproc_per_node=4 --master_port=29501 main_pretrain.py;

Fine-tuning of classification

We use NIH ChestX-ray as an example:

cd CLS-NIH_ChestX-ray
CUDA_VISIBLE_DEVICES=0 python train.py --pretrained_path "./pretrained-model/checkpoint-30.pth";
python test.py --model pretrained-model --gpu 4;

Fine-tuning of segmentation

cd Siim_Segmentation
chmod a+x ft.sh
./ft.sh
chmod a+x test.sh
./test.sh

Zero-shot phase-grounding

python MaCo_grounding.py