Eliminating Feature Ambiguity for Few-Shot Segmentation

This repository contains the code for our ECCV 2024 paper "Eliminating Feature Ambiguity for Few-Shot Segmentation", where we design a plug-in network AENet for three existing cross attention-based baselines: CyCTR, SCCAN and HDMNet.

Dependencies

Please follow the official guidelines of the selected baselines to create their virtual environments. Note that:

• You can follow SCCAN to create a shared environment for both CyCTR and SCCAN, but remember to run the following commands to install Deformable Attention for CyCTR:

  > cd SCCAN_CyCTR/model/ops/
  > bash make.sh
  

• You need to follow HDMNet to create a separate environment for HDMNet.

Directory Structure

The directory structure is:

../
├── SCCAN_CyCTR/  # code for SCCAN and CyCTR
├── HDMNet/       # code for HDMNet
├── lists/        # shared data lists
├── initmodel/    # shared pretrained backbones
└── data/         # shared data
    ├── base_annotation/
    ├── VOCdevkit2012/
    │   └── VOC2012/
    └── MSCOCO2014/           
        ├── annotations/
        ├── train2014/
        └── val2014/

Datasets

• PASCAL-5ⁱ: VOC2012 + SBD
• COCO-20ⁱ: COCO2014

You can download the pre-processed PASCAL-5ⁱ and COCO-20ⁱ datasets here, and extract them into data/ folder. Then, you need to create a symbolic link to the pascal/VOCdevkit data folder as follows:

> ln -s <absolute_path>/data/pascal/VOCdevkit <absolute_path>/data/VOCdevkit2012

Data Lists

• Download the data lists from here and put them into the lists/ directory.
• Three baselines share the same data lists, you can put the lists/ folder in the root directory, and create symbolic links as follows:

> ln -s <absolute_path>/lists <absolute_path>/SCCAN_CyCTR/lists
> ln -s <absolute_path>/lists <absolute_path>/HDMNet/lists

Backbones

• Download the pretrained backbones from here and put them into the initmodel/ directory.
• Three baselines share the same pretrained backbones, you can put the initmodel/ in the root directory, and create symbolic links as follows:

> ln -s <absolute_path>/initmodel <absolute_path>/SCCAN_CyCTR/initmodel
> ln -s <absolute_path>/initmodel <absolute_path>/HDMNet/initmodel

Pretrained Models

• Download exp_cyctr.zip to obtain CyCTR-related pretrained models, and extract them to SCCAN_CyCTR/exp.
• Download exp_sccan.zip to obtain SCCAN-related pretrained models, and extract them to SCCAN_CyCTR/exp.
• Download exp_hdmnet.zip to obtain HDMNet-related pretrained models, and extract them to HDMNet/exp.

Training and Testing Commands

• Training (with 4 GPUs)
  • Take SCCANPlus as an example:

    > cd SCCAN_CyCTR
    
    # ============================================================
    # Args for train.sh:
    #   exp_name: split0/split1/split2/split3 - different folds
    #   dataset:  pascal/coco - two benchmark datasets
    #   port:     1234 - port for DDP
    #   arch:     SCCANPlus/CyCTRPlus - model names
    #   net:      vgg/resnet50 - pretrained backbones
    #   postfix:  ddp/ddp_5s - 1 or 5 shot
    # ============================================================
    > sh train.sh split0 pascal 1234 SCCANPlus resnet50 ddp
    

• Testing (with 1 GPUs)
  • Take SCCANPlus as an example:

    > cd SCCAN_CyCTR
    > sh test.sh split0 pascal SCCANPlus resnet50 ddp
    

References

This repo is mainly built based on BAM and SCCAN. Thanks for their great work!