Feature-prompting GBMSeg: One Shot Reference Guided Training-Free Feature Matching for Glomerular Basement Membrane Segmentation and Quantification

Xueyu Liu, Guangze Shi, Rui Wang, Yexin Lai, Jianan Zhang, Lele Sun, Quan Yang, Yongfei Wu*, Weixia Han, Ming Li, and Wen Zheng
¹Taiyuan University of Technology,   ²The Second Affiliated Hospital of Shanxi Medical University，  ³Shanxi Provincial People's Hospital

🚀🚀This work has been accepted by MICCAI2024!🚀🚀

We present GBMSeg, a training-free framework that automates the segmentation and measurement of the glomerular basement membrane (GBM) in TEM using only one-shot reference images. GBMSeg leverages the robust feature matching capabilities of pretrained foundation models (PFMs) to generate initial prompts, designs novel prompting engineering for optimized prompting methods, and utilizes a class-agnostic segmentation model to obtain the final segmentation result.

Usage

Setup

• Cuda 12.0
• Python 3.9.18
• PyTorch 2.0.0

Datasets

../                          # parent directory
├── ./data                   # data path
│   ├── reference_image      # the one-shot reference image
│   ├── reference_mask       # the one-shot reference mask
│   ├── target_image         # testing images

Generate prompt

cd GBMSeg/feature-matching
python generate_prompt.py

Automatic prompt engineering

cd GBMSeg/tools
python automatic_prompt_engineering.py

Segmentation

mkdir GBMSeg/results
cd GBMSeg/segmenting-anything
python segment.py

Citation

If you find this project useful in your research, please consider citing:

@article{liu2024feature,
  title={Feature-prompting GBMSeg: One-Shot Reference Guided Training-Free Prompt Engineering for Glomerular Basement Membrane Segmentation},
  author={Liu, Xueyu and Shi, Guangze and Wang, Rui and Lai, Yexin and Zhang, Jianan and Sun, Lele and Yang, Quan and Wu, Yongfei and Li, MIng and Han, Weixia and others},
  journal={arXiv preprint arXiv:2406.16271},
  year={2024}
}

Acknowledgement

Thanks DINOv2, SAM. for serving as building blocks of GBMSeg.