Skip to content
official implementation of "Visualization of Convolutional Neural Networks for Monocular Depth Estimation"
Python
Branch: master
Clone or download
Latest commit b2ad275 Jul 24, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
figs Add files via upload Apr 10, 2019
models Add files via upload Apr 9, 2019
LICENSE Create LICENSE Apr 10, 2019
README.md Update README.md Jul 24, 2019
loaddata.py Add files via upload Apr 9, 2019
net_mask.py Add files via upload Apr 9, 2019
nyu_transform.py Add files via upload Apr 9, 2019
sobel.py Update sobel.py Apr 10, 2019
test.py Add files via upload Apr 9, 2019
train.py Add files via upload Apr 9, 2019
util.py Add files via upload Apr 9, 2019

README.md

Visualization of Convolutional Neural Networks for Monocular Depth Estimation


Junjie Hu, Yan Zhang, Takayuki Okatani http://arxiv.org/abs/1904.03380

Introduction

We attempt to interpret CNNs for monocular depth estimation. To this end, we propose to locate the most relevant pixels of input image to depth inference. We formulate it as an optimization problem of identifying the smallest number of image pixels from which the CNN can estimate a depth map with the minimum difference from the estimate from the entire image.

Predicted Masks

Extensive experimental results show

  • The behaviour of CNNs that they seem to select edges in input images depending not on their strengths but on importance for inference of scene geometry.

  • The tendency of attending not only on the boundary but the inside region of each individual object.

  • The importance of image regions around the vanishing points for depth estimation on outdoor scenes.

Please check our paper for more details.

Dependencies

  • python 2.7
  • pytorch 3.1

Running

Download the trained networks for depth estimation : Depth estimation networks

Download the trained networks for mask prediction : Mask prediction network

Download the NYU-v2 dataset: NYU-v2 dataset

  • Test

    python test.py
  • Training

    python train.py

Citation

If you use the code or the pre-processed data, please cite:

@inproceedings{Hu2019VisualizationOC,
  title={Visualization of Convolutional Neural Networks for Monocular Depth Estimation},
  author={Junjie Hu and Yan Zhang and Takayuki Okatani},
  booktitle={IEEE International Conf. on Computer Vision (ICCV)},
  year={2019}
}

@inproceedings{Hu2018RevisitingSI,
  title={Revisiting Single Image Depth Estimation: Toward Higher Resolution Maps With Accurate Object Boundaries},
  author={Junjie Hu and Mete Ozay and Yan Zhang and Takayuki Okatani},
  booktitle={IEEE Winter Conf. on Applications of Computer Vision (WACV)},
  year={2019}
}
You can’t perform that action at this time.