Meta-USCL

This is the repository for Meta UltraSound Contrastive Learning (Meta-USCL), implemented with PyTorch.

1. Brief Introduction to this Method

SimCLR + Positive Pair Interpolator (PPI, to generate positive pairs from an ultrasound video clip) + Contrastive Meta Weight Net (CMW-Net, a meta-learning-driven positive pair weighting network).

Meta-USCL (TMI paper) is an improved version of USCL (MICCAI paper). The key difference is the new CMW-Net, which is a generic method for weighting the random generated positive pairs (they tend to have highly diverse benefit for learning semantic consistency).

2. Framework of Meta-USCL

3. The Learned Weighting Scheme

(1) For different subsets (domains) of our pre-training dataset US-4, CMW-Net gives different weight distributions.

(2) The illustration of weights and similarities between an initial frame and other frames. Red boxes highlight the changing semantic marks (e.g., hepatic vein, portal vein and inferior vena cava) from the initial frame, green boxes denote the textural dissimilarities (gas in the lungs and a small portal vein section) in the following frames.

4. Quick Start

4.1 Fine-tuning directly with the pre-trained model

1. Download the 5 fold cross validation POCUS dataset
2. Run the demo with

# Using 64x64 pre-trained model to fine-tune on 64x64 image size
python eval_pretrained_model_on_POCUS/train.py --path model_ckpt/resnet18_64.pth --input_shape 64

# Using 224x224 pre-trained model to fine-tune on 224x224 image size
python eval_pretrained_model_on_POCUS/train.py --path model_ckpt/resnet18_224.pth --input_shape 224

Name	Pre-trained size	Epochs	Project head	Classifier	Accuracy	Download
ShuffleNet v2	64x64	300	Yes	Yes	87.3%
ShuffleNet v2	224x224	300	Yes	Yes	90.3%
ResNet-18	64x64	300	Yes	Yes	90.9%	link
ResNet-18	224x224	300	Yes	Yes	94.6%	link

4.2 Train Your Own Model

1. Download the Butterfly (Baidu pan Pwd:butt, Google drive) dataset
2. Train the USCL model with

python train.py

5. Apply Pre-trained Models with Mask-RCNN for Tumor Segmentation

1. Download pre-trained ResNet-18 (same as above).
2. Unzip the UDIAT-B breast tumor segmentation dataset at eval_mask_rcnn_on_UDIATB/UDIAT_Dataset_B.zip.
3. Fine-tune pre-trained model with

python eval_mask_rcnn_on_UDIATB/run.py --dataset_dir eval_mask_rcnn_on_UDIATB/UDIAT_Dataset_B --path model_ckpt/resnet18_224.pth

6. Environment

The code is developed with an Intel(R) Xeon(R) Silver 4210R CPU @ 2.40GHz and a single Nvidia Tesla V100 GPU.

The install script requirements.txt has been tested on an Ubuntu 18.04 system.

☁️☁️☁️☁️☁️☁️☁️☁️☁️☁️☁️☁️☁️☁️☁️☁️☁️☁️☁️☁️☁️☁️☁️☁️☁️☁️☁️☁️☁️☁️☁️☁️☁️☁️☁️☁️

7. License

Licensed under an MIT license.

8. Citation

This work is published at IEEE Transaction on Medical Imaging. If you find the code and dataset useful in your research, please consider citing:

@article{chen2022generating,
    title={Generating and Weighting Semantically Consistent Sample Pairs for Ultrasound Contrastive Learning},
    author={Chen, Yixiong and Zhang, Chunhui and Ding, Chris HQ and Liu, Li},
    journal={IEEE Transactions on Medical Imaging},
    year={2022},
    publisher={IEEE}
}

@inproceedings{Chen2021MICCAI,
    title={USCL: Pretraining Deep Ultrasound Image Diagnosis Model through Video Contrastive Representation Learning},
    author = {Yixiong Chen, and Chunhui Zhang, and Li Liu, and Cheng Feng, and Changfeng Dong, and Yongfang Luo, and Xiang Wan},
    journal = {MICCAI},
    year = {2021}
  }


@article{born2021accelerating,
    title={Accelerating detection of lung pathologies with explainable ultrasound image analysis},
    author={Born, Jannis and Wiedemann, Nina and Cossio, Manuel and Buhre, Charlotte and Br{\"a}ndle, Gabriel and Leidermann, Konstantin and Aujayeb, Avinash and Moor, Michael and Rieck, Bastian and Borgwardt, Karsten},
    journal={Applied Sciences},
    pages={672},
    year={2021},
}

9. Contact

Feedbacks and comments are welcome! Feel free to contact us via yixiongchen@link.cuhk.edu.cn or andyzhangchunhui@gmail.com or liuli@cuhk.edu.cn.

Enjoy!