Skip to content
master
Switch branches/tags
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
 
 
 
 
 
 

RDFNet:RGB-D Multi-level Residual Feature Fusion for Indoor Semantic Segmentation

This is the implementation of the models and test code for the "RDFNet:RGB-D Multi-level Residual Feature Fusion for Indoor Semantic Segmentation", ICCV2017.

File description

  • caffe-master: caffe used in our experiments
  • test.py: demo code
  • Each of NYU-50 / NYU-101 / NYU-152 directory includes RDF model and its prototxt corresponding to different number of resnet layers. (*You may need to change the 'nyud_dir' parameter in the prototxt.)
  • data: test data
  • nyud_layers.py: input python layer
  • gupta-utils-HHA: HHA generation utils by Gupta et al. [2]

Usage

  • Install Opencv
  • Compile pycaffe: modify the "Makefile.config" in caffe-master for your environment.
  • Download the model files.
  • Run test.py
    • Change 'caffe_root'
    • Set the 'scale' and 'model' to test.
    • To achieve the same accuracy reported in our paper, you need to implement multi-scale (0.6~1.2) ensemble as described in the paper.

Environment

Our experiments were mainly performed on Ubuntu 14.04 with CUDA7.0 / CUDNNv4 / Titan X (maxwell) / Opencv2.7

Note

  • Similarly to RefineNet,
    • Our implementation uses bicubic resize function to resize feature map.
    • We remove white boundaries of the images in NYUDv2.
  • Any comment for improvement is welcome as the code is not fully optimized. but please note that further maintenance will be infrequently performed.
  • OOM may occur for RDF-152 with the image scale larger than 1.0 on different environtment (e.g., Titan Xp, CUDA 8.0, CUDNN v6)

Citation

  • We would like to thank Guosheng Lin [3] for invaluable help.

[1] @InProceedings{Park_2017_ICCV, author = {Park, Seong-Jin and Hong, Ki-Sang and Lee, Seungyong}, title = {RDFNet: RGB-D Multi-Level Residual Feature Fusion for Indoor Semantic Segmentation}, booktitle = {The IEEE International Conference on Computer Vision (ICCV)}, month = {Oct}, year = {2017} }

[2] @incollection{guptaECCV14, author = {Saurabh Gupta and Ross Girshick and Pablo Arbelaez and Jitendra Malik}, title = {Learning Rich Features from {RGB-D} Images for Object Detection and Segmentation}, booktitle = ECCV, year = {2014}, }

[3] @inproceedings{lin2017refinenet, title={Refinenet: Multi-path refinement networks for high-resolution semantic segmentation}, author={Lin, Guosheng and Milan, Anton and Shen, Chunhua and Reid, Ian}, booktitle={IEEE Conference on Computer Vision and Pattern Recognition (CVPR)}, year={2017} }

License

For academic usage, the code is released under the permissive BSD license. For any commercial purpose, please contact the authors.

About

No description, website, or topics provided.

Resources

Releases

No releases published

Packages

No packages published

Languages