Black-Box Test-Time Shape REFINEment for Single View 3D Reconstruction

Intro

This repo contains the code used for the CVPRW 2022 paper, "Black-Box Test-Time Shape REFINEment for Single View 3D Reconstruction". For more details, please refer to the project website.

The Jupyter Notebook Simulate_Full_Pipeline.ipynb is useful for getting started, and provides code for refining several examples.

Dependencies

The following python packages are required:

• tqdm
• PIL
• numpy
• matplotlib
• trimesh
• pandas
• torch
• pytorch3d
• torchvision
• scipy
• openCV
• sklearn

For a precise list of the configuration used in the paper, refer to packages listed in the refine_env.yml file. You can also create a conda environment using it with:

conda env create --file refine_env.yml

Dataset

The 3D Object Domain Dataset Suite (3D-ODDS) is a hierarchical multiview, multidomain image dataset with 3D meshes that was created for rigoriously evaluting the effectinveness of REFINE. To download, use one of the links below (zip archive password is cvpr2659).

• Download
• Download Mirror

Related Work

The following materials were used for experiments in the paper:

• ShapeNet https://shapenet.org/
• Pix3d http://pix3d.csail.mit.edu/
• Pix2Mesh. Used under the Apache License 2.0. Commit 7c5a7a1. https://github.com/nywang16/Pixel2Mesh
• Pix2Vox. Used under the MIT License, copyright 2018 Haozhe Xie. Commit f1b8282. https://github.com/hzxie/Pix2Vox
• AtlasNet. Used under the MIT License, copyright 2019 ThibaultGROUEIX. Commit 22a0504. https://github.com/ThibaultGROUEIX/AtlasNet
• Occupancy Networks. Used under the MIT License, copyright 2019 Lars Mescheder, Michael Oechsle, Michael Niemeyer, Andreas Geiger, Sebastian Nowozin. Commit 406f794. https://github.com/autonomousvision/occupancy_networks

F-score and chamfer-L2 is built in. For the EMD and 3D IoU metrics, please refer to the following repositories.

• https://github.com/daerduoCarey/PyTorchEMD
• https://gist.github.com/LMescheder/b5e03ffd1bf8a0dfbb984cacc8c99532
• https://github.com/autonomousvision/occupancy_networks/blob/master/im2mesh/eval.py

Citations

If you use this code for your research, please consider citing:

@InProceedings{Leung_2022_CVPR,
		author = {Leung, Brandon and Ho, Chih-Hui and Vasconcelos, Nuno},
		title = {Black-Box Test-Time Shape REFINEment for Single View 3D Reconstruction},
		booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
		month = {June},
		year = {2022}
		}