Skip to content
Switch branches/tags
Go to file

Latest commit


Git stats


Failed to load latest commit information.
Latest commit message
Commit time

Deep Generative Modeling of LiDAR data (IROS 2019)

Code for reproducing all results in our paper, which can be found here
Additional results can be found here

(key) Requirements

  • Pytorch 0.4.1/0.4.0
  • TensorboardX
  • Mayavi (for display only)


├── Evaluation 
    ├──             # Evaluation of conditional generation, using Chamfer and EMD metrics
    ├──         # Generate point clouds from a pretrained model, both for clean and corrupted input        
├── Launch Scripts          
    ├──           # launch the hyperparameter search used for the proposed VAE models
    ├──      # launch the hyperparameter search used for the baseline models
├── Trained Models  
    ├── ....                # contains the weights of the models (ours and baseline) used in the paper
├── Training Logs
    ├── ....                # Contains all the tensorboard logs for all the model hyperparameter searches
├──               # Training file for GAN model
├──               # Training file for the VAE models
├──               # Implementation of all the model architectures used in the paper
├──                # Utilies for preprocessing and visualitation 

Models are logged according to the following file sturcture, where the root is specified using the --base_dir flag

├── <base_dir>
    ├── TB/
        ├── ...             # Contains the Tensorboard logs 
    ├── models/  
        ├── ...             # Contains the saved model weights, stored as `.pth` files.
├── args.json               # List of all the hyperparameters used for training.     

Train a model

We provide the full list of commands to replicate all of our results.

VAE experiments

The general command is python <flags> e.g.
python --z_dim=256 --batch_size=64 --kl_warmup_epochs=100

To get more information regarding the different flags, you can run python --help
To replicate the results from a specific model, simply provide the hyperparemeter values listed in trained_models/<model_you_want>/args.json

GAN experiments

Similarly, you can train a GAN using the following command python <flags> e.g.
python --optim=Adam --batch_size=64 --loss=1

To get more information regarding the different flags, you can run python --help

Evaluate a model

To evaluate a AE/VAE on the clean, noisy and corrupted tasks, run
python evaluation/ <path_to_base_dir_of_trained_model> <epoch_#_to_load> <emd/chamfer> e.g.
python evaluation/ trained_models/ae_xyz 209 chamfer,
which will print the reconstruction results and the noise std /missing data percentage

Examine training logs

Additionally, we provide the tensorboard logs for all models trained in the hyperparameter search. To see them in tensorboard, run
tensorboard --logdir=trained_models/

Additional Results


Our model learns a compressed representation of the input. Here we encode into a 512 dimensional vector, (60x smaller than original input) and decode it back.

Original Lidar Reconstruction from compressed (60x smaller) encoding

Surprisingly, our approch is highly robust to noisy input. Here we added gaussian noise on the input (shown on the left) rendering the lidar uninformative to the human eye. Yet, the model is able to reconstruct the point cloud with little performance loss. We note that the model was not trained on noisy data :O

Noisy input Reconstruction from noisy input

Here we repeat the same corruption process, but with even more noise

Noisy input Reconstruction from noisy input

samples from GAN model


Thanks to Fxia2 for his NNDistance module. Thanks to Thibault GROUEIX and Panos Achlioptas for open sourcing their code.
Thanks to Alexia JM for her open source code on Relativistic GANs. Please check out her work if you are working with GANs!
Thanks to Alexandre Bachaalani for his video editing help!


If you find this code useful please cite us in your work.

  author={L. {Caccia} and H. v. {Hoof} and A. {Courville} and J. {Pineau}},
  booktitle={2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)}, 
  title={Deep Generative Modeling of LiDAR Data}, 


Code for "Deep Generative Models for LiDAR Data"



No releases published


No packages published