Skip to content
No description, website, or topics provided.
Python
Branch: master
Clone or download
Latest commit 5e13e97 Mar 31, 2018
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
NYU-101 Update Link to RDF101-model Mar 26, 2018
NYU-152 Update Link to RDF152-model Mar 26, 2018
NYU-50 Update Link to RDF50-model Mar 26, 2018
data Delete test.txt~ Mar 26, 2018
README.md Update README.md Mar 30, 2018
caffe-master.tar Add files via upload Mar 26, 2018
gupta-utils-HHA.zip Add files via upload Mar 26, 2018
nyud_layers.py Update nyud_layers.py Mar 26, 2018
test.py Add files via upload Mar 26, 2018

README.md

RDFNet:RGB-D Multi-level Residual Feature Fusion for Indoor Semantic Segmentation

This is the implementation of the models and test code for the "RDFNet:RGB-D Multi-level Residual Feature Fusion for Indoor Semantic Segmentation", ICCV2017.

File description

  • caffe-master: caffe used in our experiments
  • test.py: demo code
  • Each of NYU-50 / NYU-101 / NYU-152 directory includes RDF model and its prototxt corresponding to different number of resnet layers. (*You may need to change the 'nyud_dir' parameter in the prototxt.)
  • data: test data
  • nyud_layers.py: input python layer
  • gupta-utils-HHA: HHA generation utils by Gupta et al. [2]

Usage

  • Install Opencv
  • Compile pycaffe: modify the "Makefile.config" in caffe-master for your environment.
  • Download the model files.
  • Run test.py
    • Change 'caffe_root'
    • Set the 'scale' and 'model' to test.
    • To achieve the same accuracy reported in our paper, you need to implement multi-scale (0.6~1.2) ensemble as described in the paper.

Environment

Our experiments were mainly performed on Ubuntu 14.04 with CUDA7.0 / CUDNNv4 / Titan X (maxwell) / Opencv2.7

Note

  • Similarly to RefineNet,
    • Our implementation uses bicubic resize function to resize feature map.
    • We remove white boundaries of the images in NYUDv2.
  • Any comment for improvement is welcome as the code is not fully optimized. but please note that further maintenance will be infrequently performed.
  • OOM may occur for RDF-152 with the image scale larger than 1.0 on different environtment (e.g., Titan Xp, CUDA 8.0, CUDNN v6)

Citation

  • We would like to thank Guosheng Lin [3] for invaluable help.

[1] @InProceedings{Park_2017_ICCV, author = {Park, Seong-Jin and Hong, Ki-Sang and Lee, Seungyong}, title = {RDFNet: RGB-D Multi-Level Residual Feature Fusion for Indoor Semantic Segmentation}, booktitle = {The IEEE International Conference on Computer Vision (ICCV)}, month = {Oct}, year = {2017} }

[2] @incollection{guptaECCV14, author = {Saurabh Gupta and Ross Girshick and Pablo Arbelaez and Jitendra Malik}, title = {Learning Rich Features from {RGB-D} Images for Object Detection and Segmentation}, booktitle = ECCV, year = {2014}, }

[3] @inproceedings{lin2017refinenet, title={Refinenet: Multi-path refinement networks for high-resolution semantic segmentation}, author={Lin, Guosheng and Milan, Anton and Shen, Chunhua and Reid, Ian}, booktitle={IEEE Conference on Computer Vision and Pattern Recognition (CVPR)}, year={2017} }

License

For academic usage, the code is released under the permissive BSD license. For any commercial purpose, please contact the authors.

You can’t perform that action at this time.