Leveraging Model Interpretability and Stability to increase Model Robustness

This is the backbone code of the paper Leveraging Model Interpretability and Stability to increase Model Robustness.

• The purpose of this work is to detect potential prediction errors of a CNN and cancel those predictions.
• Inputs whose prediction are cancelled are not further processed by the CNN. If the area of application allows it (medical diagnosis, malware detection, autonomous driving), those inputs can be processed by a supplementary system (human specialist, radar/lidar).

Summary

1. Overview
2. Experimental results
3. CIFAR10 file how to use
4. Imagenet file how to use

1. Overview

• The Conductance and the Label Change Rate (LCR) of a prediction are metrics used to determine if the prediction is likely to be wrong or correct.
• Conductance rely on model interpretability and check if the prediction result was obtained by emphasizing corresponding model feature maps.
• Label Change Rate rely on the stability of the prediction to model parameter changes to determine if the prediction is wrong or not.
• After a CNN (IncepionV3) made a prediction on an input image, both LCR and Conductance can be extracted from the process to examine whether the prediction is wrong or not.

2. Experimental results

• Performance (measured by AUROC) of LCR, Conductance, both LCR & Conductance to distinguish wrong and correct predictions of 3 CNN trained over 3 datasets (CIFAR10, 50 classes ImageNet, 100 classes ImageNet) are displayed in the following graphs:
• Results of using LCR & Conductance to classify test set predictions of the 3 CNN (trained over the 3 respective datasets) are displayed in the following table:

3. CIFAR10 file how to use

• Follow the 'conductance&mutation.ipynb' notebook to see how conductance and LCR of data are calculated and used to train an error detector to differentiate wrong and correct predictions of a CNN.
• Or compile the following colab notebook to see how the whole pipeline works.
• To build modified CNNs (mutants) that will be used to calculate LCR run

python cifar10/scripts/build_mutations_cifar.py

4. Imagenet file how to use

• To calculate the conductance on the training set run

python imagenet/scripts/training_conductance.py --num_class=<num_class> 

• To calculate the conductance on the validation and test set run

python imagenet/scripts/val_test_conductance.py --num_class=<num_class>

• To build modified CNNs (mutants) that will be used to calculate LCR run

python imagenet/scripts/build_mutations.py --num_class=<num_class> --limit=<limit> --gamma=<gamma> --mode=<mode>

• To calculate the LCR with these modified CNNs run

python imagenet/scripts/mutant_predistions.py --num_class=<num_class>

• 'num_class' is either 50 or 100 (default 100), it corresponds to either operate on the 50-classes Imagenet dataset or the 100-classes Imagenet dataset.
• 'limit' is the total number of modified CNN to create, default to 100.
• 'gamma' is the percentage of model weights to modify, default to 0.002.
• 'mode' is the type of operation to apply for modifying the weights. It is either 'NAI' (Neuron Activation Inverse) or 'GF' (Gaussian Fuzz). default to 'NAI'.\

After computing conductance (with or not LCR) follow code in '100_class_imagenet.ipynb' (or 50) notebook to see how the binary classifier is trained and evaluated.

work in progress to make the repository's code easier to read.