This repository is the code release of our AAAI'20 paper, "Robust Adversarial Objects against Deep Learning Models". The paper can be found here.
In this work, we proposed an adversarial attack against PointNet++, a 3D point cloud classifier, while considering creating 3D physical objects. We tried to make sure that the point clouds we generate can produce watertight objects, which can be used for 3D printing. Also we evaluated the adversarial examples with several existing defense mechanisms for 3D data introduced in previous work. For more information, please refer to our paper.
The code is tested under Python 3.7 with Tensorflow 1.13.2 in Ubuntu 18.04.
- Tensorflow < 2.0.0. Usually you can install it by one of the following command:
$ pip3 install tensorflow-gpu<2.0.0
$ conda install tensorflow-gpu<2.0.0 # (if you use Anaconda)
Or with specific version:
$ pip3 install tensorflow-gpu==1.13.2
$ conda install tensorflow-gpu==1.13.2 # (if you use Anaconda)
We recommend installing the one with GPU support for better performance.
- trimesh (used for re-sampling points from meshes)
$ pip3 install trimesh
- Other well-known stuffs such as numpy, scipy, scikit-learn, tqdm. Some of them will be installed automatically as dependencies when you install Tensorflow.
$ pip3 install numpy scipy scikit-learn tqdm
Navigate to tf_ops
directory and run compile_all.sh
.
$ cd tf_ops && sh compile_all.sh
If there are files/executables that can not be found correctly, you might have to modify the path in the script (such as the path to nvcc, which usually comes with the NVIDIA Cuda installation).
We used the modified dataset (ModelNet40)
which are re-sampled by the PointNet++ authors, which can
be downloaded here.
For more information, see here.
Move the uncompressed data folder to $PROJECT_ROOT/data/modelnet40_normal_resampled
.
We have already provided the trained models used in our paper in graph/
directory, which are
PointNet++ with Single Scale Grouping (SSG). If you want to train your own model, please refer to
PointNet++ repository for more information.
Most of the code for attacking can be found in attack.py
. To run the attack,
you need to choose the attack target, the input point cloud, and then simply type:
$ python3 attack.py --model ./graph/ssg-1024.pb --target <TARGET> --outputdir <OUTPUT> <PATH_TO_INPUT_FILE>
For example,
$ python3 attack.py --model ./graph/ssg-1024.pb --target flower_pot --outputdir outputs ./data/modelnet40_normal_resampled/bottle/bottle_0001.txt
For more information, run python3 attack.py --help
.
Most of the inference code is written in inference.py
. To inference the model,
run:
$ python3 inference.py --model ./graph/ssg-1024.pb --rounds <TIMES_TO_INFERENCE> <PATH_TO_INPUT_FILE>
For example,
$ python3 inference.py --model ./graph/ssg-1024.pb --rounds 16 ./data/modelnet40_normal_resampled/bottle/bottle_0001.txt
For more information, run python3 inference.py --help
.
Randomly pick a subset of points from input point clouds for inference. Add --rr
to the inference command.
For example,
$ python3 inference.py --model ./graph/ssg-1024.pb --rounds 16 --rr ./data/modelnet40_normal_resampled/bottle/bottle_0001.txt
Remove outlier points using kNN distance. Add --knn
to the inference command with two arguments: k
and alpha
.
For example,
$ python3 inference.py --model ./graph/ssg-1024.pb --rounds 16 --rr --knn 5 0.1 ./data/modelnet40_normal_resampled/bottle/bottle_0001.txt
For this defense, the threshold are calculated using the original data (given in the ModelNet40 dataset). The threshold can be found automatically by the following command:
$ python3 statistic_defense.py --model ./graph/ssg-1024.pb --stat <SIGMA_SQUARE> --clean ./data/modelnet40_normal_resampled/ <PATH_TO_INPUT_FILE>
For example,
$ python3 statistic_defense.py --model ./graph/ssg-1024.pb --stat 0.001 --clean ./data/modelnet40_normal_resampled/ ./data/modelnet40_normal_resampled/bottle/bottle_0001.txt
Or you can specify the threshold directly:
$ python3 statistic_defense.py --model ./graph/ssg-1024.pb --stat 0.001 --threshold 0.005 ./data/modelnet40_normal_resampled/ ./data/modelnet40_normal_resampled/bottle/bottle_0001.txt
For more information, run python3 statistic_defense.py --help
.
We use the Screened Poisson Surface Reconstruction,
which is introduced by Michael Kazhdan and Hugues Hoppe, and we choose the implementation provided by
the MeshLab software. The official website can be found here.
The reconstruction functionality can be found at Filters > Remeshing, Simplification and Reconstruction > Screened Surface Reconstruction
from the menu bar at the top of the main window. Or you can try the one released by the author of the reconstruction algorithm:
https://github.com/mkazhdan/PoissonRecon.
The resampling is done using the Trimesh package with the following simple code in Python:
For closest sampling described in paper:
import meshio
import trimesh
def closest_sample():
adv_pc = meshio.loadmesh('/path/to/adversarial/point/cloud')
recon_mesh = trimesh.load('/path/to/reconstructed/mesh', file_type='ply') # Change file_type if necessary
closest, _, _ = trimesh.proximity.closest_point(recon_mesh, adv_pc)
return closest
For random sampling:
import trimesh
def random_sample():
recon_mesh = trimesh.load('/path/to/reconstructed/mesh', file_type='ply') # Change file_type if necessary
randomly, _ = trimesh.sample.sample_surface(recon_mesh, count=10000)
return randomly
This work is based on PointNet++, which is introduced by Charles R. Qi, Li (Eric) Yi, Hao Su, Leonidas J. Guibas from Stanford University.