Neural 3D Mesh Renderer (CVPR 2018)

This repo is modified from the original pytorch implementation of Neural 3D Mesh Renderer, as well as the original chainer implementation.

The main modifications are the followings:

1. move it to Pytorch 1.5+;

2. access to the face_index_map and weight_index_map.

Requirements

Python >= 3.6 and PyTorch >= 1.5

Installation

In install mode python3 setup.py install

or editing mode pip install -v -e .

Running examples

python ./examples/example1.py
python ./examples/example2.py
python ./examples/example3.py
python ./examples/example4.py

Example 1: Drawing an object from multiple viewpoints

Example 2: Optimizing vertices

Transforming the silhouette of a teapot into a rectangle. The loss function is the difference between the rendered image and the reference image.

Reference image, optimization, and the result.

Example 3: Optimizing textures

Matching the color of a teapot with a reference image.

Reference image, result.

Example 4: Finding camera parameters

The derivative of images with respect to camera pose can be computed through this renderer. In this example the position of the camera is optimized by gradient descent.

From left to right: reference image, initial state, and optimization process.

Citation

@InProceedings{kato2018renderer
    title={Neural 3D Mesh Renderer},
    author={Kato, Hiroharu and Ushiku, Yoshitaka and Harada, Tatsuya},
    booktitle={The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
    year={2018}
}