DeepXplore code release
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
Drebin first commit Oct 9, 2017
Driving first commit Oct 9, 2017
ImageNet some new results Oct 10, 2017
MNIST some new results Oct 10, 2017
PDF first commit Oct 9, 2017
.gitignore m Oct 9, 2017
LICENSE Create LICENSE Nov 8, 2017
README.md Update README.md Nov 8, 2017

README.md

DeepXplore: Systematic DNN testing (SOSP'17)

See the SOSP'17 paper DeepXplore: Automated Whitebox Testing of Deep Learning Systems for more details.

Prerequisite

Python

The code should be run using python 2.7.12, Tensorflow 1.3.0, Keras 2.0.8, PIL, h5py, and opencv-python

Tensorflow

sudo pip install tensorflow

if you have gpu,

pip install tensorflow-gpu

Keras

pip install keras

To set Keras backend to be tensorflow (two options):

1. Modify ~/.keras/keras.json by setting "backend": "tensorflow"
2. KERAS_BACKEND=tensorflow python gen_diff.py

PIL

pip install Pillow

h5py

pip install h5py

opencv-python

pip install opencv-python

Mimicus

Install from here.

File structure

  • MNIST - MNIST dataset.
  • ImageNet - ImageNet dataset.
  • Driving - Udacity self-driving car dataset.
  • PDF - Benign/malicious PDFs captured from VirusTotal/Contagio/Google provided by Mimicus.
  • Drebin - Drebin Android malware dataset.

To run

In every directory

python gen_diff.py

Note

The trained weights are provided in each directory (if required). Drebin's weights are not part of this repo as they are too large to be hosted on GitHub. Download from here and put them in ./Drebin/.

Note that as DeepXplore use randomness for its exploration, you should fix the seed of the random number generator if you want deterministic and reproducable results. An example is shown below.

import numpy as np
import random

random.seed(1)
np.random.seed(1)

Coming soon

How to test your own DNN models.