A Flexible Neural Renderer for Material Visualization

ArXiv

Aakash KT¹, Parikshit Sakurikar¹, Saurabh Saini¹, P J Narayanan¹

¹CVIT, IIIT Hyderabad

---

[Project page] [Final Version] [arXiv] [Data] [Video] [bibtex]

---

Abstract
Photo realism in computer generated imagery is crucially dependent on how well an artist is able to recreate real-world materials in the scene. The workflow for material modeling and editing typically involves manual tweaking of material parameters and uses a standard path tracing engine for visual feedback. A lot of time may be spent in iterative selection and rendering of materials at an appropriate quality. In this work, we propose a convolutional neural network based workflow which quickly generates high-quality ray traced material visualizations on a shaderball. Our novel architecture allows for control over environment lighting and assists material selection along with the ability to render spatially-varying materials. Additionally, our network enables control over environment lighting which gives an artist more freedom and provides better visualization of the rendered material. Comparison with state-of-the-art denoising and neural rendering techniques suggests that our neural renderer performs faster and better. We provide a interactive visualization tool and release our training dataset to foster further research in this area.

Acknowledgements
We thank all the reviewers of SIGGRAPH Asia 2019, for their valuable comments and suggestions.

Running the code

Prerequisites

This code was tested on UBuntu 18.04, with Python 2.7.

Python 2.7 dependencies:
tensorflow-1.3.1
torch-1.1.0 (pytorch)
torchvision-0.0.2.post3
flask-1.0.2
pillow-5.1.0
numpy-1.13.3
opencv-python-4.1.0.25

Steps to run

• Clone this repo (git clone https://github.com/AakashKT/NeuralMaterialVisualization/).
• Download Blender 2.8 from https://www.blender.org/download/ . Extract the contents and place them inside the 'blender-2.80' directory.
• Download the network weights from: https://drive.google.com/drive/folders/1DXcVPr-g7H5SefmrOSs3xRGdMof0SBwZ?usp=sharing and place the '.pt' file inside this directory.
• Run the interactive tool with: python main.py .
• The tool will be available at localhost:8000 .

Training Data

The training dataset is available at https://drive.google.com/drive/folders/1DXcVPr-g7H5SefmrOSs3xRGdMof0SBwZ?usp=sharing