DSV-LFS: Unifying LLM-Driven Semantic Cues with Visual Features for Robust Few-Shot Segmentation

Installation

Check requirements.txt file for packages

🔧Get Started

** follow these steps to train and test DSV-LFS.**

Dataset

1. Download the dataset from the following links.

• PASCAL-5ⁱ: PASCAL VOC 2012 + SBD
• COCO-20ⁱ: MSCOCO2014

SAM ViT-H weights

Download SAM ViT-H pre-trained weights from the link.

llava-v1.5-7b

Download llava-v1.5-7b model from the link.

clip-vit-large-patch14-336

Download clip-vit-large-patch14-336 model from the link.

Training

deepspeed  train.py \
  --version="PATH_TO_llava-v1.5-7b" \
  --dataset_dir='./dataset' \
  --vision_pretrained="PATH_TO_SAM" \
  --vision-tower="PATH to clip" \
  --benchmark= "pascal" or "coco" \
  --fold="0" \
  --exp_name="name"\
  --shot="1" \

Citation

If you find this project useful in your research, please consider citing:

@inproceedings{Karimi2025DSVLFS,
  title     = {DSV-LFS: Unifying LLM-Driven Semantic Cues with Visual Features for Robust Few-Shot Segmentation},
  author    = {Amin Karimi and Charalambos Poullis},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year      = {2025},
 
}

Acknowledgement

• This work is built upon the LISA and SAM.