Skip to content


Repository files navigation

RoutedFusion: Learning Real-time Depth Map Fusion

This is the official and improved implementation of the CVPR 2020 submission RoutedFusion: Learning Real-time Depth Map Fusion.

RoutedFusion is a real-time capable depth map fusion method that leverages machine learning for fusing noisy and outlier-contaminated depth maps. It consists of two neural networks components: 1) the depth routing network that performs a 2D prepocessing of the depth maps estimating a de-noised depth map as well as corresponding confidence map. 2) a depth fusion network that predicts optimal updates to the scene representation given a canonical view of the current state of the scene representation as well as the new measurement and confidence map.

If you find our code or paper useful, please consider citing

  author = {Weder, Silvan and Sch\"onberger, Johannes L. and Pollefeys, Marc and Oswald, Martin R.},
  title = {RoutedFusion: Learning Real-Time Depth Map Fusion},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  month = {June},
  year = {2020}

Prior to using the source code in a commercial application, please contact the authors.


Below you find instructions on how to use RoutedFusion as a standalone depth map fusion pipeline for training and evaluation.


For using RoutedFusion, you need to have the following installed on you machine:

  1. docker (
  2. nvidia-docker (

Data Preparation

The models are trained on the ShapeNet v1 dataset. Therefore, the data needs to be downloaded and perpared using mesh-fusion.


  1. Download ShapeNet v1 from and unzip files
  2. Download binvox from, adjust permissions and move it to /usr/bin/
  3. Our Docker or conda environment

Install Data Generation

Run the installation script scripts/

Generate Shapenet Data

Run the data generation script scripts/



There are two possible ways of installing RoutedFusion. The recommended way is to use Docker. Alternatively, you can also use a conda environment.

Clone the repo

git clone
git submodule update --init --recursive

Build the docker image

docker build . -t routed-fusion

Start and enter the container from the image

docker run -v $PATH_TO_YOUR_PREPROCESSED_DATA:/data -v $PATH_TO_SAVE_EXPERIMENTS:/experiments --gpus all  -it routed-fusion:latest

Alternatively, create the Anaconda environment

conda env create -f environment.yml
conda activate routed-fusion
bash scripts/


Once you are in the docker container you can train RoutedFusion. First, you can train the routing network. Secondly, you can train the fusion network.

Train Routing Network

python --config configs/routing/shapenet.noise.005.yaml

Train Fusion Network

without routing

python --config configs/fusion/shapenet.noise.005.yaml

with routing

python --config configs/fusion/shapenet.noise.005.yaml --routing-model $PATH_TO_YOUR_ROUTING_MODEL

Change Data Configuration For training RoutedFusion with ShapeNet using a different artificial noise model, you can simply change the input key in the config file and add the corresponding noise model to the dataset class.


You can test RoutedFusion using either the pretrained models or your own model. Furthermore, you need to define a test config specifying the test data.

test our full pretrained model

python --experiment pretrained_models/fusion/shapenet_noise_005 --test configs/tests/shapenet.routed.noise.005.yaml

test your own model

python --experiment $PATH_TO_YOUR_EXPERIMENT --test configs/tests/shapenet.routed.noise.005.yaml

Train and test RoutedFusion on your own data

In order to train and/or test RoutedFusion on your own data, you need to add a new dataset class with the same interface as shown in the ShapeNet dataset class. You need to make sure that all keys are available. Moreover, you need to write your test configuration file and you are ready to go.