Our paper has been accepted by AAAI-2026🌹! Paper is here.
Our env: python=3.10.13, cuda:11.8.
You can prepare according to the following steps:
conda create -n myenv python=3.10.13
conda activate myenv
cd {your path}
pip install -r requirements.txt
- ViT-B-16 GET (code:52fu)
python train.py --config_file configs/RGBNT201/Signal.yml
Our model's pth files are here:
| dataset | mAP | R-1 | pth |
|---|---|---|---|
| RGBNT201 | 80.3 | 85.2 | Signal_model.pth |
| RGBNT100 | 86.3 | 97.6 | Signal_model.pth |
| MSVR310 | 53.2 | 72.4 | Signal_model.pth |
python test.py --config_file configs/RGBNT201/Signal.yml
To address multi-modal object ReID challenges, we propose Signal, a selective interaction and global-local alignment framework with three components:
- Selective Interaction Module (SIM): Selects important patch tokens from multi-modal features via intra-modal and inter-modal token selection.
- Global Alignment Module (GAM): Simultaneously aligns multi-modal features by minimizing 3D polyhedra volume in gramian space.
- Local Alignment Module (LAM): Refines fine-grained alignment via deformable sampling, handling pixel-level misalignment.
- We propose a novel selective interaction and global-local alignment framework named Signal for multi-modal object ReID, which effectively addresses the challenges of background interference and multi-modal misalignment.
- We propose the Selective Interaction Module (SIM) to leverage inter-modal and intra-modal information for selecting important patch tokens, thereby mitigating background interference in multi-modal fusion.
- We propose the Global Alignment Module (GAM) to simultaneously align multi-modal features through minimizing the volume of 3D polyhedra in the gramian space.
- We propose the Local Alignment Module (LAM) to align local features in a shift-aware manner, effectively addressing pixel-level misalignment across modalities.
- Extensive experiments on three multi-modal object ReID datasets validate the effectiveness of our method.
- Thank you for your attention and interest!