Vanilla gradients is a simple model-dependent, gradient-based saliency method.
Vanilla gradients computes saliency as the gradient of the model's output with respect to the input. This measures how much a small change to an input feature impacts the model's output. If changing the feature greatly impacts the model's output, then that feature is important to the model.
References:
- Original Paper 1: Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
- Original Paper 2: Visualizing Higher-Layer Features of a Deep Network
Implementations and Tutorials:
Aliases: gradients, saliency, vanilla saliency
Example: The vanilla gradients saliency map (right) on an ImageNet image for the class corn (left) using a Inception v3. This example is from Sanity Checks for Saliency Maps.
Vanilla gradients is deterministic.
Vanilla gradients does not have any hyperparameters.
Vanilla gradients requires a differentiable model with access to the gradients.
Computing vanilla gradients takes on the order of
Vanilla gradients saliency represents the magnitude of the change in the model’s output given a small change to an input feature.
🟨 Infidelity: Vanilla gradients' infidelity is inconclusive. It generally performs worse than SHAP, integrated gradients, and smoothed saliency methods and performs on par with guided backpropagation across MNIST, ImageNet, and CIFAR-10.
🟨 Input Consistency: Vanilla gradients' input consistency is inconclusive. It was more consistent than SmoothGrad using QRNN models, similar to SmoothGrad using an LSTM model, and less consistent than integrated gradients across LSTM, QRNN, and transformer models. Evaluated using number and gender agreement feature swaps with the Syneval and Winobias datasets.
🟢 Input Invariance: Vanilla gradients passes the input invariance test. It produces identical saliency maps for two MNIST models with different constant input shifts. It computes saliency based on the model's function, so it will always be input invariant for models with identical weights.
🟨 Perturbation Testing (LeRF): Vanilla gradients' LeRF perturbation tests were inconclusive. It performed better than random across all evaluations (MNIST, CIFAR-10, ImageNet, and IMDB datasets with MLP, CNN, Inception v3, and LSTM models) but worse than all other methods. Since it discards the sign information, it considers features that negatively impact the model equally important to features that positively affect the model, so features with a significant negative contribution are not removed until late in the LeRF process.
🟨 Perturbation Testing (MoRF): Vanilla gradients' MoRF perturbation tests were inconclusive. It performed better than random across all evaluations (MNIST, CIFAR-10, ImageNet, and IMDB datasets with MLP, CNN, Inception v3, and LSTM models) but worse than all other methods. Since it discards the sign information, it considers features that negatively impact the model equally important to features that positively affect the model, so features with a significant negative contribution are added early in the MoRF process.
🟥 ROAR: Vanilla gradients fails the ROAR test. It performs worse than random saliency when applied to an ImageNet ResNet50.
🟨 Robustness: Vanilla gradients is somewhat sensitive to random noise, which causes slight changes to the saliency despite little change to the model's output. Evaluated using an MNIST CNN.
🟨 Sensitivity: Vanilla gradients' sensitivity is inconclusive. It generally performs worse than SHAP, integrated gradients, and smoothed saliency methods and performs on par with guided backpropagation across MNIST, ImageNet, and CIFAR-10.
🟢 Stability: Vanilla gradients is relatively stable, and its outputs change minimally in response to adversarial perturbations. It performs better than LIME and equivalently to integrated gradients, input x gradients, LRP, and Occlusion. Evaluated on MNIST.
🟢 Data Randomization: Vanilla gradients changes appropriately as the model is progressively randomized. Evaluated on MNIST and Fashion MNIST.
🟨 Model Contrast Score: Vanilla gradients achieves a better-than-random model contrast score but performs worse than Grad-CAM and SmoothGrad. Evaluated on the BAM image dataset.
🟢 Cascading Model Parameter Randomization: Vanilla gradients changes appropriately as the model is progressively randomized. Evaluated an ImageNet Inception V3.
🟢 Implementation Invariance: Vanilla gradients algorithmically guarantees implementation invariance. It will produce equivalent saliency for functionally equivalent models.
🟢 Independent Model Parameter Randomization: Vanilla gradients changes appropriately as the model layers are independently randomized. Evaluated an ImageNet Inception V3.
🟨 Model Consistency: Vanilla gradients' model consistency is inconclusive. In response to model compression, its saliency stayed more consistent than SmoothGrad on a QRNN model, similarly consistent to SmoothGrad on a LSTM model, and less consistent than SmoothGrad on a transformer model and integrated gradients on all three models. Evaluated using DistillBert distillation and number and gender agreement tasks on the Syneval and Winobias datasets.
🟢 Model Weight Randomization: On a randomized model, vanilla gradients saliency is random. Evaluated on SIIM-ACR Pneumothorax and RSNA Pneumonia medical images.
🟥 Repeatability: Vanilla gradients fails repeatability. Its saliency values for two models trained in the same way are more dissimilar than similar. Evaluated on SIIM-ACR Pneumothorax and RSNA Pneumonia medical images.
🟥 Reproducibility: Vanilla gradients fails reproducibility. Its saliency values for two different architectures trained in the same way are more dissimilar than similar. Evaluated using Inception V3 and DenseNet-121 on SIIM-ACR Pneumothorax and RSNA Pneumonia medical images.
🟥 Visual Sharpening: Vanilla gradients appear noisy and are less visually coherent than Guided Backpropagation and SmoothGrad saliency methods. Evaluated on an ImageNet Inception V3 and MNIST CNN.
🟥 Localization Utility: Vanilla gradients fails localization utility. Its saliency values overlap less with the ground truth than a random model. Evaluated on SIIM-ACR Pneumothorax and RSNA Pneumonia medical images.
🟨 Plausibility: Vanilla gradients plausibility tests are inconclusive. It highlights human-important features more often than random for transformer and QRNN models, but performs worse than random for LSTM models and worse than integrated gradients and SmoothGrad across the majority of settings. Evaluated on number and gender agreement tasks using the Syneval and Winobias datasets.
🟢 The Pointing Game: Vanilla gradients highest saliency value is in the set of ground truth features more than other the baseline center feature, Deconvnet, LRP, and CAM. Evaluated on VOC07 and COCO using CNN, VGG16, and GoogleNet models.
BibTeX:
@inproceedings{vanillagradients1,
title={Deep Inside Convolutional Networks: {V}isualising Image Classification Models and Saliency Maps},
author={Simonyan, Karen and Vedaldi, Andrea and Zisserman, Andrew},
booktitle = {Proceedings of the International Conference on Learning Representations ({ICLR}), Workshop Track},
year={2014},
}
@article{vanillagradients2,
author = {Erhan, Dumitru and Bengio, Y. and Courville, Aaron and Vincent, Pascal},
year = {2009},
title = {Visualizing Higher-Layer Features of a Deep Network},
journal = {Technical Report, Univeristé de Montréal},
}