ModelNet-O: A Large-Scale Synthetic Dataset for

Occlusion-Aware Point Cloud Classification

CVIU, 2024
Zhongbin Fang · Xia Li . Xiangtai Li . Shen Zhao📧 . Mengyuan Liu📧

🙂News

• [2023.9.1] ModelNet-O and training code are open-sourced ❗❗❗
• [2023.6.9] ModelNet-O is accepted by Computer Vision and Image Understanding(CVIU). 🎉🎉🎉
• [2024.1.16] Our paper is released and GitHub repo is created ❗❗❗

⚡Hightlights

• We introduce a challenging occlusion point cloud classification dataset ModelNet-O that better reflects real-world scenarios and contains large-scale data.
• We propose a robust point cloud classification method, PointMLS, based on a multi-level sampling strategy.
• PointMLS achieves state-of-the-art overall accuracy on the occlusion point cloud dataset ModelNet-O and achieves competitive accuracy on the regular datasets, ModelNet40 and ScanObjectNN.

✋Install

Install running environment for PointMLS

conda create -n pointmls python=3.9 -y # create environment for PointMLS
conda activate pointmls
conda install pytorch==1.12.1 torchvision==0.13.1 torchaudio==0.12.1 cudatoolkit=11.3 -c pytorch # follow the Cuda version of your machine
pip install -r requirements.txt # install dependencies
pip install ninja # for c++ extensions (chamfer distance)

# install Pytorch3d for fps and knn
git clone https://github.com/facebookresearch/pytorch3d.git
cd pytorch3d
export CUB_HOME=/usr/local/cuda/include/
FORCE_CUDA=1 python setup.py install

😛ModelNet-O: a large-scale occluded point cloud dataset

Download/Generate Dataset

You can preprocess the dataset yourself, see the ModelNet/data_generate/README.md.

❗ Note that it will take a long time (about 7-10 days).

Thus, we have provided the pre-processed_datasets (recommend). Please download it and unzip it in data/

Training and testing on ModelNet-O

sh train_occluded.sh # training PointMLS in a multi-level manner
sh test_occluded.sh # testing PointMLS in a multi-level manner

Visualization

Our proposed critical point sampling method can preserve structural information of the point clouds with self-occlusion.

🧐ModelNet40 and ScanObjectNN

Usage

Training and testing on ModelNet40

python main.py --model PointMLS_basic --checkpoint "checkpoint/ModelNet40" --dataset "MN40" # training
python test.py --model PointMLS_basic --checkpoint "checkpoint/ModelNet40" --dataset "MN40" # testing
python voting.py --model PointMLS_basic

Training and testing on ScanObjectNN

python main.py --model PointMLS_basic # training
python test.py --model PointMLS_basic # testing

Acknowledgment

Our work is bulit upon previous works, thanks to the following excellent works: PointNet, PointMLP, PointView-GCN, ModelNet40, ScanObjectNN

LICENSE

Apache-2.0 license.

BibTeX

@article{fang2024modelnet,
  title={ModelNet-O: A large-scale synthetic dataset for occlusion-aware point cloud classification},
  author={Fang, Zhongbin and Li, Xia and Li, Xiangtai and Zhao, Shen and Liu, Mengyuan},
  journal={Computer Vision and Image Understanding},
  pages={104060},
  year={2024},
  publisher={Elsevier}
}