art

Jupyter Notebook Using ART to Test the Robustness of Deep Learning Models

This Jupyter notebook shows how to

• Launch training on your FfDL cluster using Jupyter notebook as a client
• Use Adversarial Robustness Toolbox (ART) to test the robustness of Deep Learning models against adversarial attacks.

Prerequisites

To run this notebook you need a Kubernetes cluster with FfDL deployed as described in the FfDL/README.md.

To store model and training data, this notebook requires access to a Cloud Object Storage (COS) instance. BlueMix Cloud Object Storage offers a free lite plan. Follow these instructions to create your COS instance and generate service credentials with HMAC keys. Then go to the COS dashboard:

• Get the cos_service_endpoint from the Endpoint tab
• In the Service credentials tab, click New Credential +
  • Add the "HMAC" inline configuration parameter: {"HMAC":true}, click Add
  • Get the access_key_id (AWS_ACCESS_KEY_ID) and secret_access_key (AWS_SECRET_ACCESS_KEY) from the cos_hmac_keys section of the instance credentials:

      "cos_hmac_keys": {
          "access_key_id": "1234567890abcdefghijklmnopqrtsuv",
          "secret_access_key": "0987654321zxywvutsrqponmlkjihgfedcba1234567890ab"
       }
    

Setup

Before running this notebook for the first time we recommend creating a Python 3 virtual environment using either virtualenv, venv (since Python 3.3), or Conda.

# assuming present working directory to be the FfDL project root
pip3 install virtualenv
virtualenv .venv/art
.venv/art/bin/pip install -r etc/notebooks/art/requirements.txt --upgrade

Export Environment Variables

Optional, this step can be done in the notebook itself.

export PUBLIC_IP=<Public IP of your FfDL cluster>
export KUBECONFIG=<Path to Kubernetes cluster configuration file>
export AWS_ACCESS_KEY_ID=<S3 cloud object storage access key ID>              
export AWS_SECRET_ACCESS_KEY=<S3 cloud object storage secret access key>
export AWS_ENDPOINT_URL=<Cloud object storage endpoint URL, i.e. 'https://s3-api.us-geo.objectstorage.softlayer.net'>
export AWS_DEFAULT_REGION=<Cloud object storage default region, i.e. 'us-east-1'>

Running the Notebook

Before starting the Jupyter notebook server, make sure to activate the Python virtual environment.

source .venv/art/bin/activate

Start the Jupyter notebook server.

jupyter-notebook --notebook-dir etc/notebooks/art
# ... use Control-C to stop the notebook server

Deactivate the virtual environment after stopping the Jupyter notebook server.

deactivate

To delete the Python virtual environment run the following command:

rm -rf .venv/art

Acknowledgements

Special thanks to Anupama-Murthi and Vijay Arya who created the original notebook which we modified here to showcase how to use ART with FfDL. If you would like to try Watson Machine Learning (WML) Service with ART check out Anupama and Vijay's notebook here:

https://github.ibm.com/robust-dlaas/ART-in-WML/Use ART to check robustness of deep learning models.ipynb