Skip to content


Folders and files

Last commit message
Last commit date

Latest commit



37 Commits

Repository files navigation


Code for paper Learning Free-Form Deformations for 3D Object Reconstruction in this repository.

Getting Started

cd /path/to/parent_dir
git clone
# non-pip dependencies
git clone                  # framework for manipulating datasets
git clone                # general 3d object utilities
git clone              # dataset access
git clone  # for chamfer loss
git clone            # optional

To run, ensure the parent directoy is on your PYTHON_PATH.

export PYTHONPATH=$PYTHONPATH:/path/to/parent_dir

So long as your PYTHONPATH is set as above, these repositories should work 'out of the box', except for tf_nearest_neighbour which requires the tensorflow op to be built. See the main repository for details.

Install pip dependencies

pip install h5py progress numpy pyemd

To use visualizations you'll also need mayavi.

pip install mayavi

See tensorflow documentation for installation. CUDA enabled GPU recommended.


This repository depends on the Dictionary Interface to Datasets (dids) repository for dataset management and util3d for various 3d utility functions.

This code base is set up to train on the ShapeNet Core dataset. We cannot provide the data for this dataset, though it is freely available for registered users. We provide functionality for rendering, loading and converting data in the shapenet repository. For this project, most data accessing should "just work". There are, however, 2 manual steps that must be completed.

  1. Add the path to your shapenet core data to the environment variable SHAPENET_CORE_PATH,
export SHAPENET_CORE_PATH=/path/to/shapenet/dataset/ShapeNetCore.v1

This folder should contain the .zip files for each category, named by the category id, e.g. all plane obj files should be in 2. Render the images,

cd /path/to/parent_dir/shapenet/core/blender_renderings/scripts
python plane
python plane

Blender is required for this. The binary must either be on your path, or supplied via's --blender_path argument.

Other data preprocessing is required before training can begin (parsing mesh data, sampling meshes, calculating FFD decomposition), though this should be handled as the need arises.

In order to evaluate IoU scores, meshes must first be converted to voxels. To allow this, make the util3d binvox binary executable

chmod +x /path/to/parent_dir/util3d/bin/binvox

You can force any of this data processing for any category to occur by manually. See the example for generating plane data below.

cd /path/to/parent_dir/shapenet/core/meshes/scripts
python plane
cd ../../point_clouds/scripts
python plane
cd ../../voxels/scripts
# For IoU data.
python plane
python plane

Note evaluation of models produces a large amount of data. In particular, inferred meshes generated for IoU evaluation can be particularly large for a low edge_length_threshold. You can safely delete any data in inference/_inferences or eval/_eval and it will be regenerated if required.


Different models can be built using different hyper-parameter sets. Models are built using the model.template_ffd_builder.TemplateFfdBuilder class. Each hyperparameter set should have a MODEL_ID and an associated model/params/MODEL_ID.json file. Default values are speficied where they are used in the code.

See paper/ for the parameter sets used for the models presented in the paper.


Training can be done via the scripts/ script. For example,

python example -s 200000

will train the model with ID 'example' for 200000 steps (default is 100000).

To view training summaries, run

tensorboard --logdir=model/_model/MODEL_ID

Training to 100000 steps as done in the paper takes roughly 8 hours to an NVidia GTX-1070.


There are a number of steps to evaluation, depending on the metrics required.

  • To create predictions (network outputs, deformation parameters Delta P), run scripts/ MODEL_ID
  • See also scripts/, scripts/ and scripts/ (slow).

Paper Figures

See the paper subdirectory for various scripts used to generate the figures presented in the paper.


If you find this code useful in your research, please cite the following paper.

  title={Learning Free-Form Deformations for 3D Object Reconstruction},
  author={Jack, Dominic and Pontes, Jhony K and Sridharan, Sridha and Fookes, Clinton and Shirazi, Sareh and Maire, Frederic and Eriksson, Anders},
  journal={arXiv preprint arXiv:1803.10932},


Since the initial release, a small bug has been fixed where batch normalization was being applied both before and after activations in some cases. This shouldn't make a massive difference to performance, but may mean models previously trained can no longer be loaded properly. To revert to older functionality, add 'use_bn_bugged_version': true to the params file.


Code for paper "Learning Free-Form Deformations for 3D Object Reconstruction"






No releases published


No packages published