Skip to content
Spherical Kernel for Efficient Graph Convolution on 3D Point Clouds
Python C++ MATLAB Cuda Other
Branch: master
Clone or download
Latest commit 27a0629 Oct 9, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
image Add files via upload Sep 23, 2019
io Add files via upload Sep 22, 2019
modelnet40_cls Add files via upload Sep 22, 2019
models Add files via upload Sep 20, 2019
post-merging Update shapenet_onehot_mIoU.m Oct 8, 2019
preprocesing Add files via upload Sep 20, 2019
ruemonge2014_seg Delete write_train_test_data.m Oct 8, 2019
s3dis_seg Add files via upload Sep 22, 2019
scannet_seg Add files via upload Sep 22, 2019
shapenet_seg Add files via upload Sep 22, 2019
tf_ops Update tf_nnquery.cpp Oct 8, 2019
utils Add files via upload Sep 20, 2019
LICENSE Initial commit Sep 20, 2019
README.md Update README.md Oct 9, 2019
compile.sh Add files via upload Sep 20, 2019

README.md

Spherical Kernel for Efficient Graph Convolution on 3D Point Clouds

Created by Huan Lei, Naveed Akhtar and Ajmal Mian

alt text

Introduction

This work is based on our Arxiv tech report, which is a significant extension of our original paper presented in IEEE CVPR2019.

We propose a spherical kernel for efficient graph convolution of 3D point clouds. Our metric-based kernels systematically quantize the local 3D space to identify distinctive geometric relationships in the data. Similar to the regular grid CNN kernels, the spherical kernel maintains translation-invariance and asymmetry properties, where the former guarantees weight sharing among similar local structures in the data and the latter facilitates fine geometric learning. The proposed kernel is applied to graph neural networks without edge-dependent filter generation, making it computationally attractive for large point clouds. In our graph networks, each vertex is associated with a single point location and edges connect the neighborhood points within a defined range. The graph gets coarsened in the network with farthest point sampling. Analogous to the standard CNNs, we define pooling and unpooling operations for our network. We demonstrate the effectiveness of the proposed spherical kernel with graph neural networks for point cloud classification and semantic segmentation using ModelNet, ShapeNet, RueMonge2014, ScanNet and S3DIS datasets.

In this repository, we release the code and trained models for classification and segmentation.

Citation

If you find our work useful in your research, please consider citing:

@article{lei2019spherical,  
  title={Spherical Kernel for Efficient Graph Convolution on 3D Point Clouds},  
  author={Lei, Huan and Akhtar, Naveed and Mian, Ajmal},  
  journal={arXiv preprint arXiv:1909.09287},  
  year={2019}  
}
@article{lei2019octree,  
  title={Octree guided CNN with Spherical Kernels for 3D Point Clouds},  
  author={Lei, Huan and Akhtar, Naveed and Mian, Ajmal},  
  journal={IEEE Conference on Computer Vision and Pattern Recognition},  
  year={2019}  
}  

License

Our code is released under MIT License (see LICENSE file for details).

Installation

Install Tensorflow. The code was tested with Python 3.5, Tensorflow 1.12.0, Cuda 9.0 and Cudnn 7.1.4 on Ubuntu 16.04. The used GPU is NVIDIA Titan XP.
**Note: while implementing the new tensorlfow operators, we assumed that the GPU supports a block of 1024 threads.

Please compile the cuda-based operations in tf-ops folder using the command

(sudo) ./compile.sh

Data Preparation

You may need to install Matlab. It is required to preprocess the datasets, such as the grid-based downsampling.
We preprocess each segmentation dataset using the corresponding function under the folder preprocessing:

preprocessing/shapenet_removeSingularPoints.m
preprocessing/ruemonge2014_prepare_data.m.m
preprocessing/scannet_prepare_data.m
preprocessing/s3dis_prepare_data.m

And then transform the *.txt files to tfrecord format for fast data feeding in Tensorflow:

cd io
python make_tfrecord_modelnet.py 
python make_tfrecord_shapenet.py  
python make_tfrecord_ruemonge2014.py   
python make_tfrecord_scannet.py  
python make_tfrecord_s3dis.py    
python make_tfrecord_s3dis_no_split.py 

Usage

All of the trained models and our results on ShapeNet and S3DIS can be downloaded from this link.

  • ModelNet

    • To train a model to classify the 40 object classes:
      cd modelnet40_cls 
      python train_modelnet.py  
      
    • To test the classification results:
      python evaluate_modelnet.py --num_votes=12  
      
  • ShapeNet

    • To train a model to segment parts of the Table Category:
      cd shapenet_seg   
      python train_shapenet.py --shape_name=Table 
      
    • To test the segmentation performance of the trained model:
      python evaluate_modelnet.py  --shape_name=Table  --model_name=xxxx    
      
  • RueMonge2014

    • train
      cd ruemonge2014_seg    
      python train_ruemonge2014.py  
      
    • test
      python evaluate_ruemonge2014.py  --model_name=xxxx    
      
  • ScanNet V2
    Download the ScanNet dataset.

    • train
      cd scannet_seg  
      python train_scannet.py  
      
    • test
      python evaluate_scannet_with_overlap.py  --model_name=xxxx    
      python scannet_block2index_with_overlap.py    
      
  • S3DIS
    Download the S3DIS dataset.

    • train
      cd s3dis_seg  
      python train_s3dis.py    
      
    • test
      python evaluate_s3dis_with_overlap.py --model_name=xxxx    
      python s3dis_block2index_with_overlap.py
      

Merging

The datasets are trained and tested with split blocks. We merge them back into complete scenes using functions under the folder post-merging in Matlab.

post-merging/scannet_merge.m
post-merging/s3dis_merge.m
You can’t perform that action at this time.