Depth-Aware Scoring and Hierarchical Alignment for Multiple Object Tracking [IEEE ICIP 2025]

Overview:

🔧 Installation

1. YOLOX & FastReID

cd external/YOLOX/
pip install -r requirements.txt && python setup.py develop

cd ../fast_reid/
pip install -r docs/requirements.txt

2. Install SAM2

git clone https://github.com/facebookresearch/sam2.git && cd sam2
pip install -e .

3. Install DepthPro

git clone https://github.com/apple/ml-depth-pro.git && cd ml-depth-pro
pip install -e .

4. Set Environment Variables

Create a .env file in your project root:

touch .env

Add the following:

SAM2_PATH="/SAM2/sam2/"
SAM2_CHECKPOINT="/SAM2/sam2/checkpoints/sam2_hiera_large.pt"
MODEL_CFG="sam2_hiera_l.yaml"
DATASET_PATH="/datasets"

5. Download Pretrained Weights

Place model weights in external/weights/:

• Deep-OC-SORT weights
• DiffMOT weights

Note: Do not untar .pth.tar YOLOX files.

---

Dataset Setup

Organize datasets as:

datasets/
├── MOT17/
│   ├── train/ 
│   └── test/
├── MOT20/
│   ├── train/ 
│   └── test/
├── dancetrack/
│   ├── train/ 
│   ├── test/ 
│   └── val/
├── sportsmot/
│   ├── train/ 
│   ├── test/ 
│   └── val/

Convert to COCO format:

python3 data/tools/convert_mot17_to_coco.py
python3 data/tools/convert_mot20_to_coco.py
python3 data/tools/convert_dance_to_coco.py
python3 data/tools/convert_sport_to_coco.py

---

Evaluation

# SportsMOT
python3 main.py --exp_name best_paper_ablations --post --grid_off --new_kf_off --cmc_off --dataset sport --w_assoc_emb 1.75 --aw_param 1

# DanceTrack
python3 main.py --exp_name best_paper_ablations --post --grid_off --new_kf_off --cmc_off --dataset dance --aspect_ratio_thresh 1000 --w_assoc_emb 1.35 --aw_param 1

# MOT17
python3 main.py --exp_name best_paper_ablations --post --grid_off --new_kf_off --cmc_off --dataset mot17 --w_assoc_emb 1.35 --aw_param 1

# MOT20
python3 main.py --exp_name best_paper_ablations --post --grid_off --new_kf_off --cmc_off --dataset mot20 --track_thresh 0.4 --iou_thresh 0.4 --w_assoc_emb 1.2 --aw_param 1

---

Acknowledgements

Built on top of:

• Deep-OC-SORT
• OC-SORT

Utilizes:

• SAM2 for segmentation
• DepthPro for zero-shot depth prediction
• YOLOX for object detection
• FastReID for Person ReIdentification

---

Citation

If you find this work useful, please cite:

@inproceedings{khanchi2025depth,
  title={Depth-aware scoring and hierarchical alignment for multiple object tracking},
  author={Khanchi, Milad and Amer, Maria and Poullis, Charalambos},
  booktitle={Proceedings of the IEEE International Conference on Image Processing (ICIP)},
  year={2025},
  note={Accepted},
  url= {https://arxiv.org/abs/2506.00774}
}