PyTorch implementation of Improved Modeling of 3D Shapes with Multi-view Depth Maps. We propose a novel encoder-decoder generative model for 3D shapes using multi-view depth maps; and show applications in single-view reconstruction, shape generation and 3D shape interpolation. Accepted as an Oral paper in 3DV.
If you use this code in your work, please cite
@inproceedings{gupta2020improved,
title={{Improved Modeling of 3D Shapes with Multi-view Depth Maps}},
author={Gupta, Kamal and Jabbireddy, Susmija and Shah, Ketul and Shrivastava, Abhinav and Zwicker, Matthias},
booktitle={International Conference on 3D Vision},
year={2020},
url={http://multiview-shapes.umiacs.io},
}
In all our experiments, we use ShapeNetCore (v2) dataset. You'll need to sign an agreement to access and download the dataset on the linked webpage.
Refer to dataset/README.md for details on data generation.
Then model training is done in two steps. In the first step, we train an encoder-decoder architecture which takes a depthmap as input to encoder. The latent representation learned by encoder is used by view-conditioned generator which generates depthmap from any given viewpoint. Note that stylegan2 generator model is borrowed from this repo.
In the second step, we train an Implicit Maximum Likelihood Expectation (IMLE) model on top of the latent codes learned in the first step.
python train_ae_avg.py /path/to/data \
--ckpt_save_directory /path/to/output/model/checkpoints \
--sample_save_directory /path/to/output/samples \
--wandb --no_noise \
--categories airplane \
--d_reg_every 0 --g_reg_every 0 \
--iter 75001 --batch 24 --num_vps 2 \
--loader_type blender --load_sil --lr 0.004 \
--channel_multiplier 1 --bins 256 --size 64 --val_batch 8 \
--use_pretrained_if_available \
--exp_name airplane_blender_base_avg_cm1_bins_256_size_64
Dump the latent codes of the model learned in the previous step and train IMLE as following
python train_imle.py
We have borrowed the code for computing CD and CMD from this repo. Please refer to the link on how to compile the code. For computing Chamfer Distance (CD):
python evaluate_recon_cd.py \
--gt_pts_path /path/to/groundtruth/pointclouds \
--gen_pts_path /path/to/reconstructed/pointclouds
For computing Earth Mover Distance (EMD):
python evaluate_recon_emd.py \
--gt_pts_path /path/to/groundtruth/pointclouds \
--gen_pts_path /path/to/reconstructed/pointclouds
MIT