An AI-based Clinician-Decision Support System for Inherited Retinal Diseases: A Multicenter Clinical Validation Trial

Contents

Table of Contents

1. Download the pretrained weights of Retina4IRD
2. Prepare the environment
3. Fine-tuning with Retina4IRD Image Model Weights
4. Fine-tuning with Retina4IRD Combined Model Weights
5. Prepare the datasets

📃Download the pretrained weights of Retina4IRD

Obtain the Retina4IRD pre-training weights on (https://huggingface.co/zycl2001/Retina4IRD)

	Source
Retina4IRD_imageModel_CFP	access
Retina4IRD_imageModel_OCT	access
Retina4IRD_combinedModel_CFP	access
Retina4IRD_combinedModel_OCT	access

🔧Prepare the environment

1. Download the project

• Open the terminal in the system.

• Clone this repository to the home directory.

git clone https://github.com/zycl2001/Retina4IRD.git

• Change the current directory to the project directory

cd /Retina4IRD

2. Prepare the running environment

1. Create environment with conda:

conda create -n retina4IRD python=3.8 -y
conda activate retina4IRD

2. Install dependencies

• Install Pytorch 1.10.0 (Cuda 11.1)

pip install torch==1.10.0+cu111 torchvision==0.11.0+cu111 torchaudio==0.10.0 -f https://download.pytorch.org/whl/torch_stable.html

• Install other Python packages

pip install -r requirements.txt

3. Download Weight

Please download Retina4IRD weights (https://huggingface.co/zycl2001/Retina4IRD) and place them in the /weights directory
/weights/image_models/Retina4IRD_CFP_checkpoint.pth
/weights/image_models/Retina4IRD_OCT_checkpoint.pth
/weights/combined_models/Retina4IRD_combinedModel_CFP.pkl
/weights/combined_models/Retina4IRD_combinedModel_OCT.pkl

📝Prepare the datasets

During fine-tuning, the images and labels of each modality are stored in CSV files: train.csv, val.csv, and test.csv, corresponding to the training, validation, and test sets, respectively. These files should be placed in the same directory. In each CSV file, cfp and oct indicate the image names for the corresponding modalities, age and other columns represent patient meta information, and gene denotes the associated hereditary eye disease. For paired-modality fine-tuning, each row contains paired data from two or more modalities.

├── csv_path
    ├──cfp
        ├──train.csv
        ├──val.csv
        ├──test.csv
    ├──oct
        ├──train.csv
        ├──val.csv
        ├──test.csv

├──image_path
    ├──cfp
    	├──1.png
    	├──2.png
    	...
    ├──oct
    	├──1.png
    	├──2.png
    	...

ID	cfp	oct	age	...	gene
1	cfp/1.jpg	oct/1.jpg	15	...	gene_1
2	cfp/2.jpg	oct/2.jpg	18	...	gene_2
...	...		...	...	....

🌱Fine-tuning with Retina4IRD Image Model Weights

1. Configuration of Image Model

You can set image model training parameters in the /cfgs/image_cls.yaml file

nb_classes: number of classes
gene: 'gene'
csv_path: Your csv_path
data_path: Your image_path
.....

2. Finetuning

The following commands fine-tune the pre-trained Retina4IRD Image Model Weights.

python run_finetuning.py \
--gene gene \
--label_column gene_label \
--weight_cfp ./weights/image_models/Retina4IRD_CFP_checkpoint.pth \
--weight_oct ./weights/image_models/Retina4IRD_OCT_checkpoint.pth \
--data_path YOUR_IMAGE_PATH \
--csv_path YOUR_CSV_PATH \
--output_dir YOUR_OUTPUT_DIR \
--epochs 50 \
--batch_size 32 \
--blr 0.005 \
--input_size 224 \
--in_domains rgb

You can also run it using the script below.

cd script/
sh target.sh

• Training scripts support YAML configuration files and command-line parameters.
• Parameters specified in the configuration file override the default settings, while command-line parameters override both the default and configured parameters.

3. Inference

You can run the script below to test the image.

python inference_retina4IRD_image_model_for_multiclass_classification.py \
--gene gene \
--test \
--label_column gene_label \
--weight_cfp ./weights/image_models/Retina4IRD_CFP_checkpoint.pth \
--weight_oct ./weights/image_models/Retina4IRD_OCT_checkpoint.pth \
--data_path YOUR_IMAGE_PATH \
--csv_path YOUR_CSV_PATH \
--output_dir YOUR_OUTPUT_DIR \
--input_size 224 \
--in_domains rgb

🌱Fine-tuning with Retina4IRD Combined Model Weights

1. Configuration of Combined Model

You can set image model training parameters in the /cfgs/combine_cls.yaml file

csv_path: Your csv_path
output_dir: Your output path
....

2. Finetuning

The following commands fine-tune the pre-trained Retina4IRD Combined Model Weights.

python myEyeTenClassfication.py \
--csv_path YOUR_CSV_PATH \
--output_dir YOUR_OUTPUT_DIR \
--combine_weight_cfp ./weights/combined_models/Retina4IRD_combinedModel_CFP.pkl \
--combine_weight_oct ./weights/combined_models/Retina4IRD_combinedModel_OCT.pkl \
--label_column gene_label \
--in_domains rgb

You can also run it using the script below.

cd script/
sh run_myEye_parms.sh

• Training scripts support YAML configuration files and command-line parameters.
• Parameters specified in the configuration file override the default settings, while command-line parameters override both the default and configured parameters.

3. Inference

After running ImageModel, you can run the following script to test the combined model.

python inference_retina4IRD_combine_model_for_multiclass_classification.py \
--test \
--csv_path YOUR_CSV_PATH \
--output_dir YOUR_OUTPUT_DIR \
--combine_weight_cfp ./weights/combined_models/Retina4IRD_combinedModel_CFP.pkl \
--combine_weight_oct ./weights/combined_models/Retina4IRD_combinedModel_OCT.pkl \
--label_column gene_label \
--in_domains rgb