XAI_SRAE_Places

This is a pytorch implementation of the Explainable Deep Networks (XNN) described in

Embedding Deep Networks into Visual Explanations (PDF)
Zhongang Qi, Saeed Khorram, Fuxin Li, in arXiv preprint arXiv:1709.05360.

For project details, please see:
http://web.engr.oregonstate.edu/~qiz/

Data and Model

Please download the data and the model from the following links:

1. The original CNN model: pytorch_vgg16_places365.pth
   
2. The input positive training data for category bedroom: ALLfeaturesbedroom.mat
   
3. The input negative training data: ALLfeaturesnegAll.mat

Quick Start

1. The original CNN model (pytorch_vgg16_places365.pth) should be put to the path: ‘./Model/’, which is converted from a pretrained caffe model (VGG16-places365) in the following link:
   https://github.com/CSAILVision/places365

2. The features of the positive training data (ALLfeatures_bedroom.mat) and the negative training data (ALLfeatures_negAll.mat) are extracted from the previously mentioned pretrained caffe model. You need to put them to the path: ‘./Model/’.

3. The images in ‘./Images/’ are from ADE20K dataset. You can download more from the following link:
   http://groups.csail.mit.edu/vision/datasets/ADE20K/

4. Run Main_run_visual_all.py to train a new XNN model, combine the trained XNN model with the original CNN model, and visualize the x-features of the combined model.

There is also a trained XNN model for the category of bedroom: SRAE_52_201803071339.pth.
You can:

• Download the trained XNN model and put it to the path: ‘./Results/52/201803071339/’,
  
• Then run ModelCombineTorch.py to combine this trained XNN model with the original CNN model,
  
• Finally run XnnVisualizeTorch.py to visualize the x-features of the combined model.

Dependencies

All code is written in Python 3.6. You will need PyTorch 0.3+ to run the excitation bp code.
The excitation bp code is from the following link:
https://github.com/greydanus/excitationbp.
Thanks Sam Greydanus for providing the excitation bp code in PyTorch.
We revised a little to the original code, thus please use the excitation bp code we provided here.