JL-DCF-pytorch

Pytorch implementation for JL-DCF: Joint Learning and Densely-Cooperative Fusion Framework for RGB-D Salient Object Detection (CVPR2020) PDF

Requirements

• Python 3.6
  
• Pytorch 1.5.0
  
• Torchvision 0.6.1
  
• Cuda 10.0

Usage

This is the Pytorch implementation of JL-DCF. It has been trained and tested on Windows (Win10 + Cuda 10 + Python 3.6 + Pytorch 1.5), and it should also work on Linux but we didn't try.

To Train

• Download the pre-trained ImageNet backbone (resnet101 and vgg_conv1, whereas the latter already exists in the folder), and put it in the 'pretrained' folder
• Download the training dataset and modify the 'train_root' and 'train_list' in the main.py
• Set 'mode' to 'train'
• Run main.py

To Test

• Download the testing dataset and have it in the 'dataset/test/' folder
• Download the already-trained JL-DCF pytorch model and modify the 'model' to its saving path in the main.py
• Modify the 'test_folder' in the main.py to the testing results saving folder you want
• Modify the 'sal_mode' to select one testing dataset (NJU2K, NLPR, STERE, RGBD135, LFSD or SIP)
• Set 'mode' to 'test'
• Run main.py

Learning curve

The training log is saved in the 'log' folder. If you want to see the learning curve, you can get it by using: tensorboard --logdir your-log-path

Pre-trained ImageNet model for training

resnet101
vgg_conv1, password: rllb

Trained model for testing

Baidu Pan: JL-DCF-pytorch, password: jdpb
Google Drive: https://drive.google.com/open?id=12u37yz-031unDPJoKaZ0goK8BtPP-6Cj

JL-DCF-pytorch saliency maps

Baidu Pan: Saliency maps, password: 4nqr
Google Drive: https://drive.google.com/open?id=1mHMN36aI5zNt50DQBivSDyYvCQ9eeGhP

Dataset

Baidu Pan:
Training dataset (with horizontal flip), password: i4mi
Testing datadet, password: 1ju8
Google Drive:
Training dataset (with horizontal flip)
Testing datadet

Performance

Below is the performance of JL-DCF-pyotrch (Pytorch implementation). Generally, the performance of Pytorch implementation is comparable to, and even slightly better than the previous Caffe implementation reported in the paper. This is probably due to the differences between deep learning platforms. Also, due to the randomness in the training process, the obtained results will fluctuate slightly.

Datasets	Metrics	Pytorch
NJU2K	S-measure	0.917
	maxF	0.919
	maxE	0.950
	MAE	0.037
NLPR	S-measure	0.931
	maxF	0.920
	maxE	0.964
	MAE	0.022
STERE	S-measure	0.906
	maxF	0.903
	maxE	0.946
	MAE	0.040
RGBD135	S-measure	0.934
	maxF	0.928
	maxE	0.967
	MAE	0.020
LFSD	S-measure	0.862
	maxF	0.861
	maxE	0.894
	MAE	0.074
SIP	S-measure	0.879
	maxF	0.889
	maxE	0.925
	MAE	0.050

Citation

Please cite our paper if you find the work useful:

    @InProceedings{Fu_2020_CVPR,
    author = {Keren Fu, Deng-Ping Fan, Ge-Peng Ji, Qijun Zhao},
    title = {JL-DCF: Joint Learning and Densely-Cooperative Fusion Framework for RGB-D Salient Object Detection},
    booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
    pages={3052--3062},
    year = {2020}
    }

Benchmark RGB-D SOD

The complete RGB-D SOD benchmark can be found in this page
http://dpfan.net/d3netbenchmark/