🌓 OpSeg: Instance Shadow Segmentation with SAM2.1

📸 Official Implementation of IJCNN 2025 Paper “OpSeg: Adapting Segment Anything Model 2 with Prompts for Efficient Instance Shadow Detection”

---

📘 简介 (Introduction)

本仓库提供了 SAM2.1_b+ 模型 在 SOBA 实例阴影分割数据集 上的微调版本。 该模型具备对物体与其阴影进行联合实例分割的能力，可作为通用视觉分割任务中对阴影敏感场景的增强版模型。

This repository provides a fine-tuned version of SAM2.1_b+ on the SOBA Instance-Shadow Segmentation Dataset, enabling joint segmentation of objects and their shadows. It serves as an enhanced foundation for general visual segmentation tasks where shadow understanding is important.

---

🧠 模型背景 (Background)

“OpSeg: Adapting Segment Anything Model 2 with Prompts for Efficient Instance Shadow Detection” (IJCNN 2025) 该仓库为论文的官方实现。我们计划在更大规模、更丰富的阴影实例数据集上继续扩展并微调其他 SAM 版本。

“OpSeg: Adapting Segment Anything Model 2 with Prompts for Efficient Instance Shadow Detection” (IJCNN 2025) This repository serves as the official implementation. We plan to extend to more diverse shadow datasets and other SAM variants.

---

🚀 性能与结果 (SOTA Performance)

据我们所知，在 SOBA 数据集的带 prompt 的测试集与挑战集上，OpSeg 取得了最优结果（参考自 MetaShadow, CVPR 2025）。

Dataset	Target	Metric	Ori.	Fz.	Ft.	↑Δ (%)
Test	Object	mIoU	0.630	0.799	0.877	+39.2
Test	Shadow	mIoU	0.236	0.554	0.763 (MetaShadow 0.710)	+223.4
Challenge	Object	mIoU	0.633	0.732	0.851	+34.4
Challenge	Shadow	mIoU	0.165	0.433	0.671	+307.4

Full Results (点击展开)

Metric	Ori.	Fz.	%Δ Fz ↑	Ft.	%Δ Ft ↑
Test - Objects
mIoU	0.630	0.799	26.7	0.877	39.2
mDice	0.694	0.850	22.5	0.917	32.1
W.IoU	0.718	0.830	15.6	0.908	26.5
W.Dice	0.779	0.877	12.6	0.942	20.9
Test - Shadows
mIoU	0.236	0.554	134.6	0.763 (MetaShadow 0.710)	223.4
mDice	0.295	0.658	123.0	0.842	185.4
W.IoU	0.167	0.546	226.8	0.781	367.4
W.Dice	0.218	0.642	194.7	0.847	289.2
Challenge - Objects
mIoU	0.633	0.732	15.6	0.851	34.4
mDice	0.707	0.803	13.5	0.901	27.4
W.IoU	0.749	0.791	5.6	0.881	17.7
W.Dice	0.817	0.857	4.8	0.922	12.8
Challenge - Shadows
mIoU	0.165	0.433	163.1	0.671	307.4
mDice	0.214	0.545	154.3	0.763	256.1
W.IoU	0.095	0.389	311.1	0.688	627.3
W.Dice	0.128	0.493	285.3	0.765	497.9

---

🛠 推荐工具 (Recommended Tool)

• ISAT（交互式半自动标注工具，整合 SAM 家族能力，包括视频追踪与多帧交互）： https://github.com/yatengLG/ISAT_with_segment_anything.git

---

💾 模型下载 (Model Download)

ModelScope

pip install modelscope

from modelscope import snapshot_download
model_dir = snapshot_download('deyang2000/SAM2_Shadow')

Google Drive 👉 Download from Google Drive

💡 在 ISAT 中使用时： 只需将下载得到的权重文件重命名为：

sam2.1_hiera_base_plus

然后放入 ISAT 工具的 checkpoints/ 目录下即可自动识别。

📚 引用 (Citation)

@INPROCEEDINGS{11228401,
  author={Li, Yanfei and Xu, Jun and Zeng, Yuan and Gong, Yi},
  booktitle={2025 International Joint Conference on Neural Networks (IJCNN)}, 
  title={OPSeg: Adapting Segment Anything Model 2 with Prompts for Efficient Instance Shadow Detection}, 
  year={2025},
  volume={},
  number={},
  pages={1-8},
  keywords={Training;Image segmentation;Visualization;Accuracy;Semantics;Object segmentation;Object detection;Inference algorithms;Robustness;Videos;Segment anything model;instance shadow detection;fine-tune},
  doi={10.1109/IJCNN64981.2025.11228401}}

---

📄 License

Apache License 2.0. See LICENSE.