Code for MICCAI2023 paper: TransLiver: A Hybrid Transformer Model for Multi-phase Liver Lesion Classification.
TransLiver is a hybrid framework with ViT backbone for liver lesion classification, which achieves an overall accuracy of 90.9% on an in-house dataset of four CT phases and seven liver lesion classes. We design a pre-processing unit to reduce the annotation cost, where we obtain lesion area on multi-phase CTs from annotations marked on a single phase. To alleviate the limitations of pure transformers, we propose a multi-stage pyramid structure and add convolutional layers to the original transformer encoder, which helps improve the model performance. We use additional cross phase tokens at the last stage to complete a multi-phase fusion, which can focus on cross-phase communication and improve the fusion effectiveness as compared with conventional modes.
We use Python 3.9.12 in our project. The main packages are included in requirements.txt.
Please prepare your own single-phase annotated data with lesion position and class. The data and its position label should be placed in /path/to/your/dataset/phase and /path/to/your/dataset/phase_label, e.g., /dataset/artery and /dataset/artery_label. The class label should be placed in /path/to/lesions, which is the directory of pre-processed lesion data.
The data and position labels should be processed to .nii.gz, and the class labels should be processed to one json file with the lesion id as the key and the class id as the value.
We have given parts of data processing codes in ./register/reg_preprocess.py, ./register/reg_postprocess.py, and ./classification/preprocess.py. You can also process your data with your own methods, but you should change the data reading mode.
The data paths and hyperparameters should be changed according to your own project. Please see ./register/config.py and ./classification/config.py.
The voxelmorph code in pytorch is in reference of VoxelMorph-torch.
./register/reg_preprocess.py: preprocess for register./register/reg_train.py: register train./register/reg_test.py: register all data./register/reg_postprocess.py: lesion matcher
./classification/preprocess.py: preprocess for classification
Get pretrain weights of CMT-S in ./pre.
Please execute ./classification/run.sh.
Please run ./classification/test.py.
@inproceedings{wang2023transliver,
title={TransLiver: A Hybrid Transformer Model for Multi-phase Liver Lesion Classification},
author={Wang, Xierui and Ying, Hanning and Xu, Xiaoyin and Cai, Xiujun and Zhang, Min},
booktitle={International Conference on Medical Image Computing and Computer-Assisted Intervention},
pages={329--338},
year={2023},
organization={Springer}
}