Skip to content
master
Switch branches/tags
Go to file
Code

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
Aug 11, 2020
Aug 11, 2020
Aug 11, 2020

README.md

RGB-D Salient Object Detection: A Survey

Authors: Tao Zhou, Deng-Ping Fan, Ming-Ming Cheng, Jianbing Shen, Ling Shao.

This is a survey to review related RGB-D SOD models along with benchmark datasets, and provide a comprehensive evaluation for these models. We also collect related review papers for SOD and light field SOD models. If you have papers to recommend or any suggestions, please feel free to contact us.

🏃 🏃 🏃 KEEP UPDATING.

alt text Fig.0: A brief chronology of RGB-D based SOD. The first early RGB-D based SOD work was the DM model, proposed in 2012. Deep learning techniques have been widely applied to RGB-D based SOD since 2017. More details can be found in our paper.



Content:

  1. Related Reviews and Surveys to SOD
  2. RGB-D SOD Models
  3. RGB-D SOD Datasets
  4. Light Field SOD
    1. LF SOD Models
    2. LF Datasets
  5. Evaluation
    1. Overall Evaluation
    2. Attribute-based Evaluation
  6. RGB-D SOD Benchmark
  7. Citation


Related Reviews and Surveys to SOD:

No. Year Pub. Title Links
01 2015 IEEE TIP Salient object detection: A benchmark Paper/Project
02 2018 IEEE TCSVT Review of visual saliency detection with comprehensive information Paper/Project
03 2018 ACM TIST A review of co-saliency detection algorithms: Fundamentals, applications, and challenges Paper/Project
04 2018 IEEE TSP Advanced deep-learning techniques for salient and category-specific object detection: A survey Paper/Project
05 2018 IJCV Attentive systems: A survey Paper/Project
06 2018 ECCV Salient Objects in Clutter: Bringing Salient Object Detection to the Foreground Paper/Project
07 2019 CVM Salient object detection: A survey Paper/Project
08 2019 IEEE TNNLS Object detection with deep learning: A review Paper/Project
09 2020 arXiv Light Field Salient Object Detection: A Review and Benchmark Paper/Project
10 2021 IEEE TPAMI Salient Object Detection in the Deep Learning Era: An In-Depth Survey Paper/Project


RGB-D SOD Models:

🔥🔥🔥Update (in 2021-02-07)

No. Year Model Pub. Title Links
🔥 106 2021 -- arXiv Self-Supervised Representation Learning for RGB-D Salient Object Detection Paper/Project
🔥 105 2021 -- Sensors Saliency Detection with Bilateral Absorbing Markov Chain Guided by Depth Information Paper/Project
🔥 104 2021 BPA-Net DSP Boundary-aware pyramid attention network for detecting salient objects in RGB-D images Paper/Project
103 2021 MobileSal arXiv MobileSal: Extremely Efficient RGB-D Salient Object Detection Paper/Project
102 2021 -- The Visual Computer A robust RGBD saliency method with improved probabilistic contrast and the global reference surface Paper/Project
101 2021 RD3D AAAI RGB-D Salient Object Detection via 3D Convolutional Neural Paper/Project
100 2021 WGI-Net CVM WGI-Net: A weighted group integration network for RGB-D salient object detection Paper/Project
99 2020 CRACE arXiv A Unified Structure for Efficient RGB and RGB-D Salient Object Detection Paper/Project
98 2020 EF-Net PR EF-Net: A novel enhancement and fusion network for RGB-D saliency detection Paper/Project
97 2020 SMAC arXiv (CVPR extension) Learning Selective Mutual Attention and Contrast for RGB-D Saliency Detection Paper/Project
96 2020 DCMF IEEE TIP RGBD Salient Object Detection via Disentangled Cross-Modal Fusion Paper/Project
95 2020 JL-DCF arXiv (CVPR extension) Siamese Network for RGB-D Salient Object Detection and Beyond Paper/Project
94 2020 MMNet ACM MM MMNet: Multi-Stage and Multi-Scale Fusion Network for RGB-D Salient Object Detection Paper/Project
93 2020 DASNet ACM MM Is depth really necessary for salient object detection? Paper/Project
92 2020 FRDT ACM MM Feature Reintegration over Differential Treatment: A Top-down and Adaptive Fusion Network for RGB-D Salient Object Detection Paper/Project
91 2020 HANet Appl. Sci. Hybrid‐Attention Network for RGB‐D Salient Object Detection Paper/Project
90 2020 DQSD IEEE TIP Depth Quality Aware Salient Object Detection Paper/Project
89 2020 DQAM arXiv Knowing Depth Quality In Advance: A Depth Quality Assessment Method For RGB-D Salient Object Detection Paper/Project
88 2020 DRLF IEEE TIP Data-Level Recombination and Lightweight Fusion Scheme for RGB-D Salient Object Detection Paper/Project
87 2020 MCINet arXiv MCINet: Multi-level Cross-modal Interaction Network for RGB-D Salient Object Detection Paper/Project
86 2020 PGAR ECCV Progressively Guided Alternate Refinement Network for RGB-D Salient Object Detection Paper/Project
85 2020 ATSA ECCV Asymmetric Two-Stream Architecture for Accurate RGB-D Saliency Detection Paper/Project
84 2020 BBS-Net ECCV BBS-Net: RGB-D Salient Object Detection with a Bifurcated Backbone Strategy Network Paper/Project
83 2020 CoNet ECCV Accurate RGB-D Salient Object Detection via Collaborative Learning Paper/Project
82 2020 DANet ECCV A Single Stream Network for Robust and Real-time RGB-D Salient Object Detection Paper/Project
81 2020 CMMS ECCV RGB-D salient object detection with cross-modality modulation and selection Paper/Project
80 2020 CAS-GNN ECCV Cascade graph neural networks for RGB-D salient object detection Paper/Project
79 2020 HDFNet ECCV Hierarchical Dynamic Filtering Network for RGB-D Salient Object Detection Paper/Project
78 2020 CMWNet ECCV Cross-modal weighting network for RGB-D salient object detection Paper/Project
77 2020 UC-Net CVPR UC-Net: Uncertainty Inspired RGB-D Saliency Detection via Conditional Variational Autoencoders Paper/Project
76 2020 S2MA CVPR Learning selective self-mutual attention for RGB-D saliency detection Paper/Project
75 2020 SSF CVPR Select, supplement and focus for RGB-D saliency detection Paper/Project
74 2020 A2dele CVPR A2dele: Adaptive and Attentive Depth Distiller for Efficient RGB-D Salient Object Detection Paper/Project
73 2020 JL-DCF CVPR JL-DCF: Joint learning and densely-cooperative fusion framework for RGB-D salient object detection Paper/Project
72 2020 D3Net IEEE TNNLS Rethinking RGB-D salient object detection: models, datasets, and large-scale benchmarks Paper/Project
71 2020 RGBS MTAP Salient object detection for RGB-D images by generative adversarial network Paper/Project
70 2020 GFNet IEEE SPL GFNet: Gate fusion network with res2net for detecting salient objects in RGB-D images Paper/Project
69 2020 SDF IEEE TIP Improved saliency detection in RGB-D images using two-phase depth estimation and selective deep fusion Paper/Project
68 2020 ICNet IEEE TIP ICNet: Information Conversion Network for RGB-D Based Salient Object Detection Paper/Project
67 2020 Triple-Net IEEE SPL Triple-complementary network for RGB-D salient object detection Paper/Project
66 2020 ASIF-Net IEEE TCYB ASIF-Net: Attention steered interweave fusion network for RGB-D salient object detection Paper/Project
65 2020 BiANet IEEE TIP Bilateral Attention Network for RGB-D Salient Object Detection Paper/Project
64 2020 PGHF IEEE Access Multi-modal weights sharing and hierarchical feature fusion for rgbd salient object detection Paper/Project
63 2020 cmSalGAN IEEE TMM cmSalGAN: RGB-D Salient Object Detection with Cross-View Generative Adversarial Networks Paper/Project
62 2020 CoCNN PR CoCNN: RGB-D deep fusion for stereoscopic salient object detection Paper/Project
61 2020 GFNet Neurocomputing A cross-modal adaptive gated fusion generative adversarial network for RGB-D salient object detection Paper/Project
60 2020 AttNet IVC Attention-guided RGBD saliency detection using appearance information Paper/Project
59 2020 SSDP arXiv Synergistic saliency and depth prediction for RGB-D saliency detection Paper/Project
58 2020 DPANet arXiv DPANet: Depth Potentiality-Aware Gated Attention Network for RGB-D Salient Object Detection Paper/Project
57 2019 DSD JVCIR Depth-aware saliency detection using convolutional neural networks Paper/Project
56 2019 DMRA ICCV Depth-induced Multi-scale Recurrent Attention Network for Saliency Detection Paper/Project
55 2019 CPFP CVPR Contrast Prior and Fluid Pyramid Integration for RGBD Salient Object Detection Paper/Project
54 2019 EPM IEEE Access Co-saliency detection for rgbd images based on effective propagation mechanism Paper/Project
53 2019 AFNet IEEE Access Adaptive Fusion for RGB-D Salient Object Detection Paper/Project
52 2019 LSF arXiv CNN-based RGB-D Salient Object Detection: Learn, Select and Fuse Paper/Project
51 2019 DGT IEEE TCYB Going from RGB to RGBD saliency: A depth-guided transformation model Paper/Project
50 2019 DCMF IEEE TCYB Discriminative cross-modal transfer learning and densely cross-level feedback fusion for RGB-D salient object detection Paper/Project
49 2019 TANet IEEE TIP Three-stream attention-aware network for RGB-D salient object detection Paper/Project
48 2019 MMCI PR Multi-modal fusion network with multi-scale multi-path and cross-modal interactions Paper/Project
47 2019 PDNet ICME Prior-model guided depth-enhanced network for salient object detection Paper/Project
46 2019 CAFM IEEE TSMC Global and Local-Contrast Guides Content-Aware Fusion for RGB-D Saliency Prediction Paper/Project
45 2019 DIL MTAP Salient object segmentation based on depth-aware image layering Paper/Project
44 2019 TSRN ICIP Two-stream refinement network for RGB-D saliency detection Paper/Project
43 2019 MLF SPL RGB-D salient object detection by a CNN with multiple layers fusion Paper/Project
42 2019 SSRC Neurocomputing Salient object detection for RGB-D image by single stream recurrent convolution neural network Paper/Project
41 2018 CDB Neurocomputing Stereoscopic saliency model using contrast and depth-guided-background prior Paper/Project
40 2018 ACCF IROS Attention-Aware Cross-Modal Cross-Level Fusion Network for RGB-D Salient Object Detection Paper/Project
39 2018 SCDL ICDSP Rgbd salient object detection using spatially coherent deep learning framework Paper/Project
38 2018 PCF CVPR Progressively complementarityaware fusion network for RGB-D salient object detection Paper/Project
37 2018 CTMF IEEE TCYB CNNs-based RGB-D saliency detection via cross-view transfer and multiview fusion Paper/Project
36 2018 ICS IEEE TIP Co-saliency detection for RGBD images based on multi-constraint feature matching and cross label propagation Paper/Project
35 2018 HSCS IEEE TMM HSCS: Hierarchical sparsity based co-saliencydetection for RGBD images Paper/Project
34 2017 ISC SIVP An integration of bottom-up and top-down salient cueson rgb-d data: saliency from objectness versus non-objectness Paper/Project
33 2017 MCLP IEEE TCYB An iterative co-saliency framework for RGBD images Paper/Project
32 2017 DF IEEE TIP RGBD Salient Object Detection via Deep Fusion Paper/Project
31 2017 MDSF IEEE TIP Depth-Aware Salient Object Detection and Segmentation via Multiscale Discriminative Saliency Fusion and Bootstrap Learning Paper/Project
30 2017 MFF IEEE SPL RGB-D saliency object detection via minimum barrier distance transformand saliency fusion Paper/Project
29 2017 TPF ICCVW A Three-Pathway Psychobiological Framework of Salient Object Detection Using Stereoscopic Technology Paper/Project
28 2017 CDCP ICCVW An innovative salient object detection using center-dark channel prior Paper/Project
27 2017 BED ICCVW Learning RGB-D Salient Object Detection using background enclosure, depth contrast, and top-down features Paper/Project
26 2017 MFLN ICCVS RGB-D Saliency Detection by Multi-stream Late Fusion Network Paper/Project
25 2017 M3Net IROS M3Net: Multi-scale multi-path multi-modal fusion network and example application to RGB-D salient object detection Paper/Project
24 2017 HOSO DICTA HOSO: Histogram of Surface Orientation for RGB-D Salient Object Detection Paper/Project
23 2016 GM ACCV Visual Saliency detection for RGB-D images with generative mode Paper/Project
22 2016 DSF ICASSP Depth-aware saliency detection using discriminative saliency fusion Paper/Project
21 2016 DCI ICASSP Saliency analysis based on depth contrast increased Paper/Project
20 2016 BF ICPR RGB-D saliency detection under Bayesian framework Paper/Project
19 2016 DCMC IEEE SPL Saliency detection for stereoscopic images based on depth confidence analysis and multiple cues fusion Paper/Project
18 2016 SE ICME Salient object detection for RGB-D image via saliency evolution Paper/Project
17 2016 LBE CVPR Local Background Enclosure for RGB-D Salient Object Detection Paper/Project
16 2016 PRC IEEE Access Improving RGBD Saliency Detection Using Progressive Region Classification and Saliency Fusion Paper/Project
15 2015 SF CAC Selective features for RGB-D saliency Paper/Project
14 2015 MGMR ICIP RGB-D saliency detection via mutual guided manifold ranking Paper/Project
13 2015 SRD ICRA Salient Regions Detection for Indoor Robots using RGB-D Data Paper/Project
12 2015 DIC TVC Depth incorporating with color improves salient object detection Paper/Project
11 2015 SFP ICIMCS Salient object detection in RGB-D image based on saliency fusion and propagation Paper/Project
10 2015 GP CVPRW Exploiting global priors for RGB-D saliency detection Paper/Project
09 2014 ACSD ICIP Depth saliency based on anisotropic center-surround difference Paper/Project
08 2014 DESM ICIMCS Depth Enhanced Saliency Detection Method Paper/Project
07 2014 LHM ECCV RGBD Salient Object Detection: A Benchmark and Algorithms Paper/Project
06 2014 SRDS ICDSP Salient region detection for stereoscopic images Paper/Project
05 2013 SOS Neurocomputing Depth really Matters: Improving Visual Salient Region Detection with Depth Paper/Project
04 2013 RC BMVC Depth really Matters: Improving Visual Salient Region Detection with Depth Paper/Project
03 2013 LS BMVC An In Depth View of Saliency Paper/Project
02 2012 RCM ICCSE Depth combined saliency detection based on region contrast model Paper/Project
01 2012 DM ECCV Depth matters: Influence of depth cues on visual saliency Paper/Project


RGB-D SOD Datasets:

No. Dataset Year Pub. Size #Obj. Types Resolution Download
1 STERE 2012 CVPR 1000 ~One Internet [251-1200] * [222-900] link
2 GIT 2013 BMVC 80 Multiple Home environment 640 * 480 link
3 DES 2014 ICIMCS 135 One Indoor 640 * 480 link
4 NLPR 2014 ECCV 1000 Multiple Indoor/outdoor 640 * 480, 480 * 640 link
5 LFSD 2014 CVPR 100 One Indoor/outdoor 360 * 360 link
6 NJUD 2014 ICIP 1985 ~One Moive/internet/photo [231-1213] * [274-828] link
7 SSD 2017 ICCVW 80 Multiple Movies 960 *1080 link
8 DUT-RGBD 2019 ICCV 1200 Multiple Indoor/outdoor 400 * 600 link
9 SIP 2020 TNNLS 929 Multiple Person in wild 992 * 774 link
10 ReDWeb-S 2020 arXiv 3179 Multiple Diversity [133-937] * [132-996] link


Light Field SOD:

LF SOD Models:

No. Year Model Pub. Title Links
01 2014 LFS CVPR Saliency detection on light field Paper/Project
02 2015 WSC CVPR A weighted sparse coding framework for saliency detection Paper/Project
03 2015 DILF IJCAI Saliency detection with a deeper investigationof light field Paper/Project
04 2016 RL ICASSP Relative location for light field saliency detection Paper/Project
05 2017 MA ACM TOMM Saliency detection on light field: A multi-cue approach Paper/Project
06 2017 BIF NPL A two-stage bayesian integration framework for salient object detection on light field Paper/Project
07 2017 LFS IEEE TPAMI Saliency Detection on Light Field Paper/Project
08 2017 RLM ICIVC Saliency detection with relative location measure in light field image Paper/Project
09 2017 SGDC CVPR Salience guided depth calibration for perceptually optimized compressive light field 3D display Paper/Project
10 2018 DCA FiO Depth-induced cellular automata for light field saliency Paper/Project
11 2019 DLLF ICCV Deep learning for light field saliency detection Paper/Project
12 2019 DLSD IJCAI Deep light-field-driven saliency detection from a single view Paper/Project
13 2019 Molf NIPS Memory-oriented decoder for light field salient object detection Paper/Project
14 2020 ERNet AAAI Exploit and replace: An asymmetrical two-stream architecture for versatile light field saliency detection Paper/Project
15 2020 DCA IEEE TIP Saliency detection via depth-induced cellular automata onlight field Paper/Project
16 2020 RDFD MTAP Region-based depth feature descriptor for saliency detection light field Paper/Project
17 2020 LFNet IEEE TIP LFNet light field fusion network for salient object detection Paper/Project
18 2020 LFDCN IEEE TIP Light field saliency detection with deep convolutional networks Paper/Project

LF Datasets:

No. Dataset Year Pub. Size Description Download
1 LFSD 2014 CVPR 100 It contains 60 indoor and 40 outdoor scenes, and most scenes consist of only one salient object link
2 HFUT 2017 ACM TOMM 255 Most scenes contain multipleobjects that appear within different locations and scales under complex background clutter link
3 HFUT 2017 ACM TOMM 255 Most scenes contain multipleobjects that appear within different locations and scales under complex background clutter link
4 DUTLF-FS 2019 ICCV 1465 It contains several challenges, including lower contrast between salient objects and cluttered background, multiple disconnected salient objects, and dark or strong light conditions link
5 DUTLF-MV 2019 IJCAI 1580 Each light field consists of multi-view images and a corresponding ground truth link
6 Lytro Illum 2020 IEEE TIP 640 It includes several challenging factors, e.g., inconsistent illumi?nation conditions, and small salient objects existing in a similar or cluttered background link


Evaluation:

Overall Evaluation:

alt text Fig.1: A comprehensive evaluation for 24 representative RGB-D based SOD models, including LHM, ACSD, DESM, GP, LBE, DCMC, SE, CDCP, CDB, DF, PCF, CTMF, CPFP, TANet, AFNet, MMCI, DMRA, D3Net, SSF, A2dele, S2MA, ICNet, JL-DCF, and UC-Net. We obtain the terms of $S_{\alpha}$ and MAE values for the 24 models on five datasets (i.e., STERE, NLPR, LFSD, DES, and SIP ). We report the mean values of $S_{\alpha}$ and MAE across the five datasets. Note that these better models are shown in the upper left corner (\ie, with a larger $S_{\alpha}$ and smaller MAE).

======================= run evaluation code ===============================

  1. We have computed values of different evaluation metrics for each image of each models and save as '***.mat', and the results can be downloaded from Google Drive or Baidu Drive(extraction code: urra).
  2. Please unzip the downloaded file 'Sal_Det_Results_24_Models.zip' and put it into the file 'results';
  3. To run 'run_overall_evaluation.m' (plot Fig.1 )

======================================================================

alt text Fig.2: PR curves for 24 RGB-D based models on STERE, NLPR, LFSD, DES, SIP, GIT, SSD, and NJUD datasets.

alt text Fig.3: F-measures under different thresholds for 24 RGB-D based models on STERE, NLPR, LFSD, DES, SIP, GIT, SSD, and NJUD datasets.

======================= run plot code ===========================

  1. To run 'run_plot_curves.m' (plot Fig.2 and Fig.3)

=============================================================



Attribute-based Evaluation:



RGB-D SOD Benchmark :

The complete RGB-D SOD benchmark can be found in this page: http://dpfan.net/d3netbenchmark/



Citation:

If you find our survey paper and evaluation code are useful, please cite the following paper:

@article{zhou2020rgbd,
	title={RGB-D Salient Object Detection: A Survey},
	author={Zhou, Tao and Fan, Deng-Ping and Cheng, Ming-Ming and Shen, Jianbing and Shao, Ling},
	journal={Computational Visual Media},
	pages={1--33},
	year={2021},
	publisher={Springer}
}

Releases

No releases published

Packages

No packages published

Languages