Reposity of "Beyond Visual Cues: CoT-Enhanced Reasoning for Semi-supervised Medical Image Segmentation".
Semi-supervised medical image segmentation has emerged as a dominant research problem in medical image analysis, mitigating annotation scarcity by leveraging consistency regularization on unlabeled data. However, existing approaches operate predominantly via visual pattern matching, relying heavily on pixel-level similarities. This visual-centric dependency often falters in clinical scenarios characterized by the visual-semantic mismatch, where visually similar lesions warrant distinct diagnostic conclusions, thus failing to capture the underlying diagnostic logic used by experts. To address this, we move beyond visual cues and propose CERS (CoT-Enhanced Reasoning Segmentation), a framework that integrates Chain-of-Thought (CoT) reasoning to distinguish pathologically distinct cases. Specifically, we construct a knowledge pool enriched with linguistic reasoning descriptions generated by large language models (LLMs). A semantic-aware reference selection strategy is introduced to identify historical evidence, filtering candidates first by morphology, and then refining them via CoT consistency to eliminate hard negatives. Furthermore, a multi-scale coordinate attention module (MCAM) is designed to effectively fuse this reasoning-derived context into the decoding process. Extensive experiments demonstrate the superiority of CERS against state-of-the-art approaches, particularly in resolving boundary ambiguities and semantic inconsistencies.
python=3.12 and using the following command:
pip install -r requirements.txt
The original data can be downloaded in following links:
This dataset is organized in a structured directory format to ensure clear separation between images, annotations, and metadata. The overall folder structure is as follows:
brisc2025/
├── frames/
│ ├── brisc2025_test_00001_gl_ax_t1.png
│ ├── brisc2025_test_00002_gl_ax_t1.png
│ └── ...
│
├── masks/
│ ├── brisc2025_test_00001_gl_ax_t1.png
│ ├── brisc2025_test_00002_gl_ax_t1.png
│ └── ...
│
├── reports/
│ ├── combined_messages_mask.json
│ ├── combined_messages_no_mask.json
│ ├── cot_output_gpt5_mask.csv
│ └── cot_output_gpt5_no_mask.csv
│
└── text.csv
The text annotation of MosMedData+ and QaTa-COV19 follows LViT. Before warming up, run the following command to generate CoTs:
python generate_cot/generate_cot.py
Notice: CoT and text description of three datasets we used is now updated. See the folder CoT.
Before training, you can warm up to get a better initailized parameters by the following command:
python warm_up_train_two_decoder_clip.py
Train the model by this command:
rag_train_two_decoder.py