Skip to content


Repository files navigation

Deep Multi-view Depth Estimation with Predicted Uncertainty


In this paper, we address the problem of estimating dense depth from a sequence of images using deep neural networks. Specifically, we employ a dense-optical-flow network to compute correspondences and then triangulate the point cloud to obtain an initial depth map. Parts of the point cloud, however, may be less accurate than others due to lack of common observations or small baseline-to-depth ratio. To further increase the triangulation accuracy, we introduce a depth-refinement network (DRN) that optimizes the initial depth map based on the image’s contextual cues. In particular, the DRN contains an iterative refinement module (IRM) that improves the depth accuracy over iterations by refining the deep features. Lastly, the DRN also predicts the uncertainty in the refined depths, which is desirable in applications such as measurement selection for scene reconstruction. We show experimentally that our algorithm outperforms state-of-the-art approaches in terms of depth accuracy, and verify that our predicted uncertainty is highly correlated to the actual depth error.

Setting up Environment

For convenience, the code are assumed to be run inside NVIDIA-Docker. For instructions on installing NVIDIA-Docker, please follow the following steps (note that this is for Ubuntu 18.04):

For more detailed instructions, please refer to this link.

  1. Install Docker

    sudo apt-get update
    sudo apt-get install apt-transport-https ca-certificates curl gnupg-agent \
    curl -fsSL | sudo apt-key add -
    sudo apt-key fingerprint 0EBFCD88
    sudo add-apt-repository \
       "deb [arch=amd64] \
       $(lsb_release -cs) \
    sudo apt-get update
    sudo apt-get install docker-ce docker-ce-cli

    To verify Docker installation, run:

    sudo docker run hello-world
  2. Install NVIDIA-Docker

    curl -s -L | \
      sudo apt-key add -
    distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
    curl -s -L$distribution/nvidia-docker.list | \
      sudo tee /etc/apt/sources.list.d/nvidia-docker.list
    sudo apt-get update
    curl -s -L | \
      sudo apt-key add -
    sudo apt-get install nvidia-docker2
    sudo pkill -SIGHUP dockerd

To activate the docker environment, run the following command:

nvidia-docker run -it --rm --ipc=host -v /:/home

where / is the directory in the local machine (in this case, the root folder), and /home is the reflection of that directory in the docker. This has also specified NVIDIA-Docker with PyTorch version 20.03 which is required to ensure the compatibility between the packages used in the code (at the time of submission).

Inside the docker, change the working directory to this repository:

cd /home/PATH/TO/THIS/REPO/DeepMultiviewDepth

Run Demo

  1. Please download and extract the files provided by this link to ./checkpoints directory.

  2. Run to get the results of the demo dataset in ./viusalization folder.

The results contain inputs, intermediate outputs, and final outputs of the proposed pipeline, same as the figure below: Alt text


No description, website, or topics provided.






No releases published


No packages published