Skip to content

zwbx/DFM-Net

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

79 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

DFM-Net (ACM MM 2021)

Official repository for paper Depth Quality-Inspired Feature Manipulation for Efficient RGB-D Salient Object Detection | δΈ­ζ–‡η‰ˆ

News


Block diagram of DFM-Net.

The most efficient RGB-D SOD method ⚑

  • Low model size: Model size is only 8.5Mb, being 6.7/3.1 smaller than the latest lightest models A2dele and MobileSal.
  • High accuracy: SOTA performance on 9 datasets (NJU2K, NLPR, STERE, RGBD135, LFSD, SIP, DUT-RGBD, RedWeb-S, COME).
  • High Speed: Cost 50ms on CPU (Core i7-8700 CPU), being 2.9/2.4 faster than the latest fastest models A2dele and MobileSal.


Performance visualization. Performance visualization. The vertical axis indicates the average S-measure over six widely used datasets (NJU2K, NLPR, STERE, RGBD135, LFSD, SIP). The horizontal axis indicates CPU speed. The circle area is proportional to the model size.

Extension πŸ”₯

Depth Quality-Inspired Feature Manipulation for Efficient RGB-D and Video Salient Object Detection

  • More comprehensive comparison:
    • Benchmark results on DUT-RGBD, RedWeb-S, COME are updated.
    • Metric of maximum-batch inference speed is added.
    • We re-test the inference speed of ours and compared methods on Ubuntu 16.04.
  • Working mechanism explanation
    • Further analyses verify the ability of DQFM in distinguishing depth maps of various qualities without any quality labels.
  • Application on efficient VSOD
    • One of the lightest VSOD methods!
    • Joint training strategy is proposed.

Easy-to-use to boost your RGB-D SOD network

If you use a depth branch as an affiliate to the RGB branch:

  • Use DQW/DHA to boost performance with extra 0.007/0.042Mb model size increased
  • Use our light-weight depth backbone to improve efficiency

if you adopt parallel encoders for RGB and depth:

Test

Directly run test.py

The test maps will be saved to './resutls/'.

data preparation

Results

  • We provide testing results of 9 datasets (NJU2K, NLPR, STERE, RGBD135, LFSD, SIP, DUT-RGBD πŸ†•, RedWeb-S πŸ†•, COME πŸ†•).

  • Evaluate the result maps: You can evaluate the result maps using the tool in Matlab Version or Python_GPU Version.

  • Note that the parameter file is 8.9Mb, which is 0.4Mb bigger than we report in the paper because keys denoting parameter names also occupy some space. Then put them under the following directory:

     -dataset\ 
       -RGBD_train
       -NJU2K\  
       -NLPR\
       ...
     -pretrain
       -DFMNet_300_epoch.pth
       ...
    

Training

  • Download training data(eb2z)
  • Modify setting in options.py and run train.py

Application on VSOD πŸ†•

Citation

Please cite the following paper if you use this repository in your research

@inproceedings{zhang2021depth,
title={Depth quality-inspired feature manipulation for efficient RGB-D salient object detection},
author={Zhang, Wenbo and Ji, Ge-Peng and Wang, Zhuo and Fu, Keren and Zhao, Qijun},
booktitle={Proceedings of the 29th ACM International Conference on Multimedia},
pages={731--740},
year={2021}
}

@artical{zhang2022depth,
title={Depth Quality-Inspired Feature Manipulation for Efficient RGB-D and Video Salient Object Detection},
author={Zhang, Wenbo and  Fu, Keren and  Wang, Zhuo and Ji, Ge-Peng and Zhao, Qijun},
booktitle={arXiv:2208.03918},
year={2022}
}

About

No description, website, or topics provided.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages