Skip to content
Point cloud classification in Real-time using 3DmFV representation and CNNs
Python Cuda C++ Shell
Branch: master
Clone or download
sitzikbs Added files for EMD and joint loss for decoder.
Removed transform_net unnecessary dependency.
Updated citation ref.
Cleanup.
Latest commit 74c3f78 Nov 26, 2018
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
doc Updating teaser image Jul 2, 2018
models Added files for EMD and joint loss for decoder. Nov 26, 2018
utils Added files for EMD and joint loss for decoder. Nov 26, 2018
LICENSE.txt First Commit Jul 2, 2018
README.md Added files for EMD and joint loss for decoder. Nov 26, 2018
provider.py First Commit Jul 2, 2018
train_cls.py First Commit Jul 2, 2018
train_decoder.py First Commit Jul 2, 2018

README.md

3DmFV : Three-Dimensional Point Cloud Classification in Real-Time using Convolutional Neural Networks

Created by Yizhak Ben-Shabat (Itzik), Michael Lindenbaum, and Anath Fischer from Technion, I.I.T.

3DmFV Architecture

Introduction

This is the code for training a point cloud classification network using 3D modified Fisher Vectors.

This work will be presented in IROS 2018 in Madrid, Spain and will also be published in Robotics and Automation Letters.

Modern robotic systems are often equipped with a direct 3D data acquisition device, e.g. LiDAR, which provides a rich 3D point cloud representation of the surroundings. This representation is commonly used for obstacle avoidance and mapping. Here, we propose a new approach for using point clouds for another critical robotic capability, semantic understanding of the environment (i.e. object classification). Convolutional neural networks (CNN), that perform extremely well for object classification in 2D images, are not easily extendible to 3D point clouds analysis. It is not straightforward due to point clouds' irregular format and a varying number of points. The common solution of transforming the point cloud data into a 3D voxel grid needs to address severe accuracy vs memory size tradeoffs. In this paper we propose a novel, intuitively interpretable, 3D point cloud representation called 3D Modified Fisher Vectors (3DmFV). Our representation is hybrid as it combines a coarse discrete grid structure with continuous generalized Fisher vectors. Using the grid enables us to design a new CNN architecture for real-time point cloud classification. In a series of performance analysis experiments, we demonstrate competitive results or even better than state-of-the-art on challenging benchmark datasets while maintaining robustness to various data corruptions.

Citation

If you find our work useful in your research, please cite our work:

@article{ben20183dmfv,
  title={3DmFV: Three-Dimensional Point Cloud Classification in Real-Time Using Convolutional Neural Networks},
  author={Ben-Shabat, Yizhak and Lindenbaum, Michael and Fischer, Anath},
  journal={IEEE Robotics and Automation Letters},
  volume={3},
  number={4},
  pages={3145--3152},
  year={2018},
  publisher={IEEE}
}

Preprint:

@article{ben20173d,
  title={3D Point Cloud Classification and Segmentation using 3D Modified Fisher Vector Representation for Convolutional Neural Networks},
  author={Ben-Shabat, Yizhak and Lindenbaum, Michael and Fischer, Anath},
  journal={arXiv preprint arXiv:1711.08241},
  year={2017}
}

Installation

Install Tensorflow. You will also need to install h5py, scikit-learn.

The code was tested with Python 2.7, TensorFlow 1.2.1, CUDA 8.0.61 and cuDNN 5105 on Ubuntu 16.04. The code was reported by users to have also been tested on Windows 10 with python 3.6, tensorflow 1.12, cuda 9.0.176 and cudnn 9.0.

This code uses the infrastructure of PointNet as a template, however, many substantial changes have been made to the CNN model and point cloud representation.

Download the ModelNet40 data in this link.

Classification

Train the point cloud classification model using the default settings on ModelNet40, using:

python train_cls.py

Alternatively, you can tweak the different GMM parameters (e.g. number of gaussians ) or learning parameters (e.g. learning rate) using

python train_cls.py  --gpu=0 --log_dir='log' --batch_size=64 --num_point=1024 --num_gaussians=8 --gmm_variance=0.0156 
--gmm_type='grid' --learning_rate=0.001  --model='voxnet_pfv' --max_epoch=200 --momentum=0.9 --optimizer='adam'
 --decay_step=200000  --weight_decay=0.0 --decay_rate=0.7

The model will be saved to log directory. Consecutive runs with the same directory name will be saved in numbered subdirectories in order to prevent accidental overwrite of trained models.

License

Our code is released under MIT License (see LICENSE file for details).

Disclaimer

I am a mechanical engineer, not a software engineer. git is relatively new to me. Therefore, if you find any place I have made an error or have an improvement recommendation, I would appreciate your advice.

You can’t perform that action at this time.