DCMF

Learning Discriminative Cross-modality Features for RGB-D Saliency Detection (DCMF)

source code for our TIP 2021 paper "Learning Discriminative Cross-modality Features for RGB-D Saliency Detection" by Fengyun Wang, Jinshan Pan, Shoukun Xu, and Jinhui Tang (PDF)

created by Fengyun Wang, email: fereenwong@gmail.com

Requirement

1. Pytorch 1.7.0 (a lower vision may also workable)
2. Torchvision 0.7.0

Data Preparation

Training: with 1400 images from NJU2K, 650 images from NLPR, and 800 images from DUT-RGBD (And 100 images from NJU2K and 50 images from NLPR for validation).

Testing: with 485 images from NJU2K, 300 images from NLPR, 400 images from DUT-RGBD, 1000 images from STERE, 1000 images from ReDWeb-S, 100 images from LFSD, and 80 images from SSD.

You can directly download these dataset (training and testing) from here:

• NJU2K [baidu_pan fetch_code:bvrg | google_drive]
• NLPR [baidu_pan fetch_code:6a2g | google_drive]
• DUT-RGBD [baidu_pan fetch_code:hqbv | google_drive]
• STERE[baidu_pan fetch_code:ffgx | google_drive]
• ReDWeb-S [baidu_pan fetch_code:zupl | google_drive] （use testset only）
• LFSD [baidu_pan fetch_code:0vx1 | google_drive]
• SSD100 [baidu_pan fetch_code:qs2y | google_drive]

After downloading, put them into your_RGBD_Dataset folder, and it should look like this:

-- your_RGBD_Dataset
   |-- NJU2K
   |   |-- trainset
   |   |-- | RGB
   |   |-- | depth
   |   |-- | GT
   |   |-- testset
   |   |-- | RGB
   |   |-- | depth
   |   |-- | GT
   |-- STERE
   |   |-- RGB
   |   |-- depth
   |   |-- GT
   ...

Training

1. Download the pretrained VGG model [baidu pan fetch code: 44be | google drive] and put it into ./pretrained_model folder.
2. Run python train.py your_RGBD_Dataset for training.

Testing on Our Pretrained model

1. Download our pretrained model [baidu_pan fetch_code:kc76 | google_drive] and then put it in ./checkpoint folder.
2. Run python test.py ./checkpoint/corr_pac.pth your_RGBD_Dataset. The predictions will be in ./output folder.

Ours Saliency Maps

• NJU2K [baidu_pan fetch_code:hxt8 | google_drive]
• NLPR [baidu_pan fetch_code:h1oe | google_drive]
• DUT-RGBD [baidu_pan fetch_code:vni4 | google_drive]
• STERE[baidu_pan fetch_code:8su3 | google_drive]
• ReDWeb-S [baidu_pan fetch_code:27hs | google_drive]
• LFSD [baidu_pan fetch_code:vapc | google_drive]
• SSD100 [baidu_pan fetch_code:2y3i | google_drive]
• RGBD135 [baidu_pan fetch_code:jhnp | google_drive]

Citation

If you think our work is helpful, please cite

@article{wang2022learning,
  title={Learning Discriminative Cross-modality Features for RGB-D Saliency Detection},
  author={Wang, Fengyun and Pan, Jinshan and Xu, Shoukun and Tang, Jinhui},
  journal={IEEE Transactions on Image Processing},
  year={2022},
  publisher={IEEE}
}