Skip to content
PointFlow : 3D Point Cloud Generation with Continuous Normalizing Flows
Branch: master
Clone or download
Guandao Yang
Guandao Yang Debug: update encoder
Latest commit 4dda09e Jul 16, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
docs Merge branch 'master' of https://github.com/stevenygd/PointFlow Jul 1, 2019
metrics Add training/testing/demo codes Jul 14, 2019
models Debug: update encoder Jul 16, 2019
scripts Update: batch size. Jul 16, 2019
.gitignore
LICENSE Initial commit Apr 11, 2019
README.md Update README.md Jul 16, 2019
args.py Add training/testing/demo codes Jul 14, 2019
datasets.py Add training/testing/demo codes Jul 14, 2019
demo.py Add training/testing/demo codes Jul 14, 2019
install.sh Add training/testing/demo codes Jul 14, 2019
test.py
train.py Add training/testing/demo codes Jul 14, 2019
utils.py Add training/testing/demo codes Jul 14, 2019

README.md

PointFlow : 3D Point Cloud Generation with Continuous Normalizing Flows

This repository contains a PyTorch implementation of the paper:

PointFlow : 3D Point Cloud Generation with Continuous Normalizing Flows.

Guandao Yang*, Xun Huang*, Zekun Hao, Ming-Yu Liu, Serge Belongie, Bharath Hariharan

Introduction

As 3D point clouds become the representation of choice for multiple vision and graphics applications, the ability to synthesize or reconstruct high-resolution, high-fidelity point clouds becomes crucial. Despite the recent success of deep learning models in discriminative tasks of point clouds, generating point clouds remains challenging. This paper proposes a principled probabilistic framework to generate 3D point clouds by modeling them as a distribution of distributions. Specifically, we learn a two-level hierarchy of distributions where the first level is the distribution of shapes and the second level is the distribution of points given a shape. This formulation allows us to both sample shapes and sample an arbitrary number of points from a shape. Our generative model, named PointFlow, learns each level of the distribution with a continuous normalizing flow. The invertibility of normalizing flows enables computation of the likelihood during training and allows us to train our model in the variational inference framework. Empirically, we demonstrate that PointFlow achieves state-of-the-art performance in point cloud generation. We additionally show our model is able to faithfully reconstruct point clouds and learn useful representations in an unsupervised manner.

Examples

Dependencies

  • Python 3.6
  • CUDA 10.0.
  • G++ or GCC 5.
  • PyTorch. Codes are tested with version 1.0.1
  • torchdiffeq.
  • (Optional) Tensorboard for visualization of training process.

Following is the suggested way to install these dependencies:

# Create a new conda environment
conda create -n PointFlow python=3.6
conda activate PointFlow

# Install pytorch (please refer to the commend in the official website)
conda install pytorch=1.0.1 torchvision cudatoolkit=10.0 -c pytorch -y

# Install other dependencies such as torchdiffeq, structural losses, etc.
./install.sh

Dataset

The point clouds are uniformly sampled from meshes from ShapeNetCore dataset (version 2) and use the official split. Please use this link to download the ShapeNet point cloud. The point cloud should be placed into data directory.

mv ShapeNetCore.v2.PC15k.zip data/
cd data
unzip ShapeNetCore.v2.PC15k.zip

Please contact us if you need point clouds for ModelNet dataset.

Training

Example training scripts can be found in scripts/ folder.

# Train auto-encoder (no latent CNF)
./scripts/shapenet_airplane_ae.sh # Train with single GPU, about 7-8 GB GPU memory
./scripts/shapenet_airplane_ae_dist.sh # Train with multiple GPUs

# Train generative model
./scripts/shapenet_airplane_gen.sh # Train with single GPU, about 7-8 GB GPU memory 
./scripts/shapenet_airplane_gen_dist.sh # Train with multiple GPUs 

Pre-trained models and test

Pretrained models can be downloaded from this link. Following is the suggested way to evaluate the performance of the pre-trained models.

unzip pretrained_models.zip;  # This will create a folder named pretrained_models

# Evaluate the reconstruction performance of an AE trained on the airplane category
CUDA_VISIBLE_DEVICES=0 ./scripts/shapenet_airplane_ae_test.sh; 

# Evaluate the reconstruction performance of an AE trained with the whole ShapeNet
CUDA_VISIBLE_DEVICES=0 ./scripts/shapenet_all_ae_test.sh;

# Evaluate the generative performance of PointFlow trained on the airplane category.
CUDA_VISIBLE_DEVICES=0 ./scripts/shapenet_airplane_gen_test.sh

Demo

The demo relies on Open3D. Following is the suggested way to install it:

conda install -c open3d-admin open3d 

The demo will sample shapes from a pre-trained model, save those shapes under the demo folder, and visualize those point clouds. Once this dependency is in place, you can use the following script to use the demo for the pre-trained model for airplanes:

CUDA_VISIBLE_DEVICES=0 ./scripts/shapenet_airplane_demo.py

Cite

Please cite our work if you find it useful:

@article{pointflow,
 title={PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows},
 author={Yang, Guandao and Huang, Xun, and Hao, Zekun and Liu, Ming-Yu and Belongie, Serge and Hariharan, Bharath},
 journal={arXiv},
 year={2019}
}
You can’t perform that action at this time.