SILT: Shadow-aware Iterative Label Tuning for Learning to Detect Shadows from Noisy Labels

Description

This is the pytorch implementation of the ICCV 2023 paper "SILT: Shadow-aware Iterative Label Tuning for Learning to Detect Shadows from Noisy Labels" by Han Yang, Tianyu Wang, Xiaowei Hu and Chi-Wing Fu.

How to Run

1. Install dependencies

# clone project   
git clone https://github.com/Cralence/SILT.git

# create conda environment
cd SILT
conda env create -f environment.yaml
conda activate silt
pip install opencv-python
pip install omegaconf==2.3.0

2. Download the additional non-shadow dataset from here if needed. Pretrained weights for the backbone encoders can be downloaded from the table below. Then, set the correct path and whether to use the additional dataset in configs/silt_training_config.yaml. Note that we use the additional dataset only when training on SBU.

3. Train the model by running:

python train.py --dataset SBU --backbone PVT-b5

4. Test the model by running:

python infer.py --dataset SBU --ckpt path_to_weight  

Dataset

Our relabeled SBU test set can be downloaded from here.

Pretrained Model

Model	Params（M)	Pretrained Backbone	SBU	ISTD	UCF
EfficientNet-B3	12.18	-	5.23	2.00	9.18
EfficientNet-B7	67.80	-	4.62	1.46	7.97
ResNeXt-101	90.50	weight	5.08	1.53	9.27
ConvNeXt-B	100.68	-	5.11	1.15	8.62
PVT v2-B3	49.42	weight	4.36	1.11	7.25
PVT v2-B5	86.14	weight	4.19	1.16	7.23

Citation

@inproceedings{yang2023silt,
  title={SILT: Shadow-aware Iterative Label Tuning for Learning to Detect Shadows from Noisy Labels},
  author={Han Yang, Tianyu Wang, Xiaowei Hu, Chi-Wing Fu},
  booktitle={IEEE International Conference on Computer Vision},
  year={2023}
}