Skip to content


Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?


Failed to load latest commit information.
Latest commit message
Commit time
December 28, 2020 11:19
October 26, 2020 12:08
October 26, 2020 12:00
October 26, 2020 12:08
October 26, 2020 12:08
October 20, 2020 02:44
October 20, 2020 02:44
October 26, 2020 12:08
October 26, 2020 12:08
October 26, 2020 12:08
October 26, 2020 12:08
October 26, 2020 12:08
October 26, 2020 12:08
October 26, 2020 12:08

SDF-SRN: Learning Signed Distance 3D Object Reconstruction from Static Images

Chen-Hsuan Lin, Chaoyang Wang, and Simon Lucey
Advances in Neural Information Processing Systems (NeurIPS), 2020

Project page:
arXiv preprint:

We provide PyTorch code for both the ShapeNet and PASCAL3D+ experiments.


This code is developed with Python3 (python3). PyTorch 1.4+ is required.
It is recommended to install the dependencies with conda by running

conda env create --file requirements.yaml python=3

This creates a conda environment named sdfsrn-env. Activate it with

conda activate sdfsrn-env

You may want to install with virtualenv; however, this repository depends on VIGRA to compute the distance transforms, which does not seem to be pip installable. Some workarounds would include (a) installing VIGRA from source, or (b) replacing the VIGRA distance transform function with scipy.ndimage.distance_transform_edt (significantly slower).


  • ShapeNet

    Download the ShapeNet renderings of Kato et al. from the DVR repository (33GB):
    (this file is huge and takes a long time to fully unzip, so we extract only the 3 categories of interest in this work)
    unzip NMR_Dataset/02691156/* # airplane
    unzip NMR_Dataset/02958343/* # car
    unzip NMR_Dataset/03001627/* # chair
    In the data/NMR_Dataset directory, download the post-processed surface point clouds:
    tar -zxvf yvencf3ts8itfgyuh5sap9q7dy5r1elg.gz
    rm yvencf3ts8itfgyuh5sap9q7dy5r1elg.gz
    There should be a pointcloud3.npz within each shape directory, along with the original pointcloud.npz. You can check with
    ls NMR_Dataset/02691156/10155655850468db78d106ce0a280f87
    If you're interested in creating ground-truth point clouds for other object categories, please refer to the README in data.

    Download the PASCAL3D+ (v1.1) dataset under the data directory (7.7GB):
    Also under the data directory, download the object masks and ground-truth point clouds for the 3 categories (23MB):
    tar -zxvf uyz0txthw0ufjwury0f3z3iuhqdbaet9.gz
    rm uyz0txthw0ufjwury0f3z3iuhqdbaet9.gz

Pretrained models

First, create a directory to store the pretrained models:

mkdir -p pretrained

Then under pretrained, download the pretrained model(s) by running the commands

# ShapeNet (trained on multi-view renderings, 615MB each)
wget -O shapenet_airplane.ckpt  # airplane
wget -O shapenet_car.ckpt  # car
wget -O shapenet_chair.ckpt  # chair
# PASCAL3D+ (197MB each)
wget -O pascal3d_airplane.ckpt # airplane
wget -O pascal3d_car.ckpt # car
wget -O pascal3d_chair.ckpt # chair

Compiling the CUDA libraries

The Chamfer distance function can be compiled by running python3 install under external/chamfer3D. The source code is taken/modified from the AtlasNet repository.
When compiling CUDA code, you may need to modify CUDA_PATH accordingly.

Running the code

  • Evaluating the downloaded pretrained models

    # ShapeNet (trained on multi-view renderings)
    python3 --model=sdf_srn --yaml=options/shapenet/sdf_srn.yaml --name=airplane_pretrained --load=pretrained/shapenet_airplane.ckpt --tb= --visdom= --eval.vox_res=128
    python3 --model=sdf_srn --yaml=options/shapenet/sdf_srn.yaml --name=car_pretrained --load=pretrained/shapenet_car.ckpt --tb= --visdom= --eval.vox_res=128
    python3 --model=sdf_srn --yaml=options/shapenet/sdf_srn.yaml --name=chair_pretrained --load=pretrained/shapenet_chair.ckpt --tb= --visdom= --eval.vox_res=128
    # PASCAL3D+
    python3 --model=sdf_srn --yaml=options/pascal3d/sdf_srn.yaml --name=airplane_pretrained --load=pretrained/pascal3d_airplane.ckpt --tb= --visdom= --eval.vox_res=128 --eval.icp
    python3 --model=sdf_srn --yaml=options/pascal3d/sdf_srn.yaml --name=car_pretrained --load=pretrained/pascal3d_car.ckpt --tb= --visdom= --eval.vox_res=128 --eval.icp
    python3 --model=sdf_srn --yaml=options/pascal3d/sdf_srn.yaml --name=chair_pretrained --load=pretrained/pascal3d_chair.ckpt --tb= --visdom= --eval.vox_res=128 --eval.icp

    This will create the following files in the output directory (e.g. output/sdf_srn_pascal3d/car_pretrained):

    • chamfer.txt: the (bidirectional) Chamfer distance error for each example.
    • dump/*_mesh.ply: the resulting 3D meshes (from zero isosurface extraction with marching cubes).
    • dump/*.png: images including input/rendered RGB images, input/predicted masks, depth maps and surface normal maps.
    • dump/vis.html: a webpage to visualize all the images for convenience.

    The overall Chamfer distance error (the numbers reported in the paper) will also be shown on screen.
    Note that it takes longer to evaluate the PASCAL3D+ models since we run ICP to pre-align the predictions to the ground-truth shapes.

  • Training from scratch

    To train SDF-SRN, we first quickly pretrain the generator with a spherical SDF for 1000 iterations with:

    # ShapeNet
    python3 --model=sdf_srn_pretrain --yaml=options/shapenet/sdf_srn_pretrain.yaml
    # PASCAL3D+
    python3 --model=sdf_srn_pretrain --yaml=options/pascal3d/sdf_srn_pretrain.yaml

    This helps SDF-SRN converge to a feasible solution, otherwise it may get stuck in bad local minima.

    For the main training:

    # ShapeNet (~100K iterations for airplanes and cars, ~200K iterations for chairs)
    python3 --model=sdf_srn --yaml=options/shapenet/sdf_srn.yaml --name=airplane --max_epoch=24 --loss_weight.shape_silh=1
    python3 --model=sdf_srn --yaml=options/shapenet/sdf_srn.yaml --name=car --max_epoch=27
    python3 --model=sdf_srn --yaml=options/shapenet/sdf_srn.yaml --name=chair --max_epoch=28
    # PASCAL3D+ (~30K iterations)
    python3 --model=sdf_srn --yaml=options/pascal3d/sdf_srn.yaml --name=airplane --freq.eval=30 --freq.ckpt=30 --max_epoch=500
    python3 --model=sdf_srn --yaml=options/pascal3d/sdf_srn.yaml --name=car --freq.eval=10 --freq.ckpt=10 --max_epoch=170
    python3 --model=sdf_srn --yaml=options/pascal3d/sdf_srn.yaml --name=chair --freq.eval=60 --freq.ckpt=60 --max_epoch=900

    The above command for ShapeNet runs single-view training on multi-view data. To train on single-view ShapeNet data (only 1 view is available per CAD model) with the reported settings, run

    # single-view ShapeNet chairs (~50K iterations)
    python3 --model=sdf_srn --yaml=options/shapenet/sdf_srn.yaml --name=chair_1view_1kcad --data.shapenet.train_view=1 --data.train_sub=1000 --data.augment.brightness=0.2 --data.augment.contrast=0.2 --data.augment.saturation=0.2 --data.augment.hue=0.5 --freq.eval=100 --freq.ckpt=100 --max_epoch=800

    This trains on a subset of 1000 chair CAD models with 1 viewpoint each while randomly jittering the colors.

    To evaluate the trained models:

    # ShapeNet
    python3 --model=sdf_srn --yaml=options/shapenet/sdf_srn.yaml --name=airplane --tb= --visdom= --eval.vox_res=128 --resume
    python3 --model=sdf_srn --yaml=options/shapenet/sdf_srn.yaml --name=car --tb= --visdom= --eval.vox_res=128 --resume
    python3 --model=sdf_srn --yaml=options/shapenet/sdf_srn.yaml --name=chair --tb= --visdom= --eval.vox_res=128 --resume
    # PASCAL3D+
    python3 --model=sdf_srn --yaml=options/pascal3d/sdf_srn.yaml --name=airplane --tb= --visdom= --eval.vox_res=128 --eval.icp --resume
    python3 --model=sdf_srn --yaml=options/pascal3d/sdf_srn.yaml --name=car --tb= --visdom= --eval.vox_res=128 --eval.icp --resume
    python3 --model=sdf_srn --yaml=options/pascal3d/sdf_srn.yaml --name=chair --tb= --visdom= --eval.vox_res=128 --eval.icp --resume

    The expected output is similar to those described above (in the pretrained models section).

  • Visualizing the results

    We have included code to visualize the training over TensorBoard. The TensorBoard events include the following:

    • SCALARS: the losses and bidirectional Chamfer distances (for both training and validation sets).
    • IMAGES: visualization of the RGB/mask/depth/normal images.

    We also provide visualization of dense point clouds sampled on the zero isosurface in Visdom.

  • General usage of the codebase

    The simplest command to run training is:

    python3 --model=sdf_srn

    This will run model/ as the main engine with options/sdf_srn.yaml as the main config file. Note that sdf_srn is hierarchically inherited from implicit and base, which makes the codebase customizable.
    The complete configuration will be printed upon execution. To override specific options, add --<key>=value or --<key1>.<key2>=value (and so on) to the command line. The configuration will be loaded as the variable opt throughout the codebase.
    If you want to reproduce the reported results, load preset configurations with the yaml option (details below).

    Some tips on using and understanding the codebase:

    • The computation graph for forward/backprop is stored in var throughout the codebase.
    • The losses are stored in loss. To add a new loss function, just implement it in compute_loss() and add its weight to opt.loss_weight.<name>. It will automatically be added to the overall loss and logged to Tensorboard.
    • If you are using a multi-GPU machine, you can add --gpu=<gpu_number> to specify which GPU to use. Multi-GPU training/evaluation is currently not supported.
    • To resume from a previous checkpoint, add --resume=<epoch_number>, or just --resume to resume from the latest checkpoint.
    • (to be continued....)

If you find our code useful for your research, please cite

  title={SDF-SRN: Learning Signed Distance 3D Object Reconstruction from Static Images},
  author={Lin, Chen-Hsuan and Wang, Chaoyang and Lucey, Simon},
  booktitle={Advances in Neural Information Processing Systems ({NeurIPS})},

Please contact me ( if you have any questions!


SDF-SRN: Learning Signed Distance 3D Object Reconstruction from Static Images 🎯 (NeurIPS 2020)







No releases published


No packages published