Minimal Adversarial Examples for Deep Learning on 3D Point Clouds (ICCV 2021)

This repository contains a Tensorflow implementation of the paper:

Minimal Adversarial Examples for Deep Learning on 3D Point Clouds
Jaeyeon Kim, Binh-Son Hua, Duc Thanh Nguyen, Sai-Kit Yeung
Hong Kong University of Science and Technology, VinAI Research, Deakin University

Abstract: With recent developments of convolutional neural networks, deep learning for 3D point clouds has shown significant progress in various 3D scene understanding tasks, e.g., object recognition, semantic segmentation. In a safetycritical environment, it is however not well understood how such deep learning models are vulnerable to adversarial examples. In this work, we explore adversarial attacks for point cloud-based neural networks. We propose a unified formulation for adversarial point cloud generation that can generalise two different attack strategies. Our method generates adversarial examples by attacking the classification ability of point cloud-based networks while considering the perceptibility of the examples and ensuring the minimal level of point manipulations. Experimental results show that our method achieves the state-of-the-art performance with higher than 89% and 90% of attack success rate on synthetic and real-world data respectively, while manipulating only about 4% of the total points.

@InProceedings{Kim_2021_ICCV,
    author    = {Kim, Jaeyeon and Hua, Binh-Son and Nguyen, Thanh and Yeung, Sai-Kit},
    title     = {Minimal Adversarial Examples for Deep Learning on 3D Point Clouds},
    booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
    month     = {October},
    year      = {2021},
    pages     = {7797-7806}
}

Please CITE our paper whenever our model implementation is used to help produce published results or incorporated into other software.

Getting Started

The codebase is tested on

• Ubuntu
• CUDA 10.0, CuDNN 7

Installation

• Clone this repository:

git clone https://github.com/hkust-vgd/minimal_adversarial_pcd.git
cd minimal_adversarial_pcd

• Install dependencies:

conda create -n minimal_adversarial_pcd python=3.7
conda activate minimal_adversarial_pcd
conda install -c anaconda tensorflow-gpu=1.14

• For distance code, please set the CUDA path and run:

bash ./utils/tf_nndistance_compile.sh

Datasets

• CAD dataset: we use the CAD dataset with 40 categories from Modelnet40
• Scan dataset: we use real scans dataset with 15 categories from ScanObjectNN If you need to try with your own dataset, you need to make the .txt, .h5 file and the label .txt file accordingly.

Pre-trained models

Run the below command to train Pointnet

python ./pointnet/train.py

For your reference, the pretrained model is available from here
And please refer the different model codes from Pointnet++, DGCNN, SpiderCNN, PointASNL.

Attack

To run the attack, set the data, ckpt path and run:

python main.py

The descriptions of the arguments are shown below.

Args	Descriptions
dataset_type	Choose the dataset {scanobnn, modelnet40}
attack_type	Choose the dataset {perturbation, addition}
h_dist_weight	lambda for distance loss, default is 50
class_loss_weight	lambda for classification loss, default is 1
count_weight	lambda for minimum loss, default is 0.15

The code for adversarial attack is in attack.py

Visualize

We provide the code for visualization in visualize.py. Please use it with the Jupyter Interactive Window.
The sample point clouds are in test_sample folder.

Acknowledgments

Our source code is developed based on the codebase of the following papers: 3D Point Cloud Adversarial Attacks and Defenses, Generating 3D Adversarial Point Clouds.

We also specially thank the following paper authors for their work and effort to release source code, pre-trained weights, and data: Pointnet, Pointnet++, DGCNN, SpiderCNN, PointASNL, Adversarial point perturbations on 3D objects, PointCloud Saliency Maps, Robust Adversarial Objects against Deep Learning Models, ScanObjectNN, Modelnet40.

Contacts

If you have any questions, please drop an email to jaeyeon.kim@connect.ust.hk or open an issue in this repository.