MultIVAP defense against adversarial examples

This repository contains a reference implementation for our defense against adversarial examples, which uses a technique from conformal prediction called inductive Venn-ABERS predictors (IVAPs; see [1, 2, 3] for more details). This is a follow-up to our ESANN 2019 contribution where we proposed a similar technique that is limited to binary classification only [4]. Here, we extend the method to the multiclass case and improve upon its clean accuracy and adversarial robustness. The full paper is available here.

Prerequisites

We tested our code using the following dependencies:

• Python 3.6.8 (https://www.python.org/);
• TensorFlow 1.14.0 (https://www.tensorflow.org/);
• Keras 2.2.4 (https://www.keras.io/);
• NumPy 1.16.4 (http://www.numpy.org/);
• Foolbox 2.2.1 (https://foolbox.readthedocs.io);
• CVXOPT 1.2.3 (https://cvxopt.org/documentation/index.html).

There is a requirements file included, so if you have the correct Python version you should be able to install all other dependencies using pip install -r requirements.txt.

Running the code

There are two ways our code can be run: either you follow the Jupyter notebook cifar10_demo.ipynb included in this repository or you can use the command-line script. The notebook is self-explanatory; the command-line script can be invoked as follows:

python main.py cifar10 --batch_size 128 --epochs 10 --frac .8 --eta .03

The first argument is the only mandatory one and must name a Python module, in this case cifar10.py. This module must define at least the following methods:

• get_optimizer. Returns a Keras optimizer to be used for fitting the model.
• load_datasets. Returns a tuple (x_train, y_train, x_test, y_test) specifying the training and test data.
• create_model. Returns the Keras model that will be trained and used for instantiating the MultIVAP.

The other arguments are optional:

• batch_size. Specifices the batch size to use when processing sets of samples in training and inference.
• epochs. Number of epochs to train the model.
• frac. Maximum fraction of GPU memory to use.
• eta. Maximum ℓ_∞ perturbation budget for adversarial attacks.

Note that the command-line script will generate a Markdown report under reports/module.md where module is the module name (in this case, cifar10) as well as several plots in the directory plots/. You should be able to directly run our code for MNIST, Fashion-MNIST and CIFAR-10 if you have all of the dependencies and if the plots and reports directories exist. The SVHN and Asirra data sets are not included in Keras and should be downloaded separately:

• Download link for SVHN (note that we used the cropped 32x32 digits in our experiments)
• Download link for Asirra

The IVAP implementation we use here is provided by Paolo Toccaceli.

References

1. Vovk, Vladimir, Ivan Petej, and Valentina Fedorova. "Large-scale probabilistic predictors with and without guarantees of validity." Advances in Neural Information Processing Systems. 2015. PDF
2. Vovk, Vladimir, Alex Gammerman, and Glenn Shafer. Algorithmic learning in a random world. Springer Science & Business Media, 2005. PDF
3. Shafer, Glenn, and Vladimir Vovk. "A tutorial on conformal prediction." Journal of Machine Learning Research 9.Mar (2008): 371-421. PDF
4. Peck, Jonathan, Bart Goossens and Yvan Saeys. "Detecting adversarial examples with inductive Venn-ABERS predictors." European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning. 2019. PDF