Code for Fong and Vedaldi 2017, "Interpretable Explanations of Black Boxes by Meaningful Perturbation"
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.

Interpretable Explanations of Black Boxes by Meaningful Perturbation

Ruth Fong and Andrea Vedaldi




  1. Install Jianming Zhang's version of Caffe, which includes implementations for the gradient, guided backprop, and excitation backprop saliency methods.
  2. Add models from Caffe's Model Zoo (recommended: bvlc_googlenet, vgg16, bvlc_reference_caffenet), add force_backward: true to each model's deploy .prototxt, and edit paths to the models in
  3. Download ImageNet images and create imdb files for train heldout and val by running the following in a python shell after cd code/caffe/python:
    from helpers import write_imdb
    write_imdb($ILSVRC_DATASET/train, '../../../data/ilsvrc12/annotated_train_heldout.txt', '../../../data/ilsvrc12/annotated_train_heldout_imdb.txt')
    write_imdb($ILSVRC_DATASET/val, $CAFFEDIR/data/ilsvrc12/val, '../../../data/ilsvrc12/val_imdb.txt')
  4. Include your local copy of the above repo in your python path by running export PYTHONPATH=$PYTHONPATH:$CAFFEDIR (or including it in your .bash_profile or .bashrc)
  5. Run sh $CAFFEDIR/data/ilsvrc12/ to download ILSVRC support files.
  6. Change caffe_dir to $CAFFEDIR in
  7. (Optional but quite useful) Download ImageNet Annotations: (found here)
  8. (Optional, for animal parts experiment only) Download animal parts annotations from here


All pycaffe files are in code/caffe/python. Public support for matconvnet will come soon (see dev branch for some experimental code).

Learn Perturbation Masks

See (the functions optimize_mask and generate_learned_mask provide most of the functionality).

To generate masks (and debugging figures) for the the first 10 images in the heldout set, run: python train_heldout -f $FIG_DIR -m $MASK_DIR -g [$GPU]

Reproduce Figures

See figures.ipynb to reproduce all figures in the main text. Public support for reproducing all supplementary figures will come soon.

Weak Localization

See localization.ipynb to reproduce numbers in Table 1.

To produce weak localization bounding boxes, use the python OUT_PATH/OUT_DIR DATA_DESC HEATMAP_TYPE THRESHOLD_TYPE [MASK_DIR] [GPU]

  • OUT_PATH/OUT_DIR is where to save the output file with bounding box information (a directory is taken as input when DATA_DESC == annotated_train_heldout
  • DATA_DESC $\in$ [annotated_train_heldout, val, animal_parts]
  • HEATMAP_TYPE $\in$ [mask, saliency (a.k.a. gradient), guided_backprop, excitation_backprop], and
  • THRESHOLD_TYPE $\in$ [min_max_diff (a.k.a. value), energy, mean]
  • MASK_DIR is where the masks for the 50k val images are saved (only include when HEATMAP_TYPE == mask)