Skip to content
Python
Branch: master
Clone or download

Latest commit

Fetching latest commit…
Cannot retrieve the latest commit at this time.

Files

Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
doc/images
interprettensor Computational efficiency for larger models with simple and eps rules Mar 28, 2018
LICENSE
README.md
_version.py
setup.cfg
setup.py

README.md

Interpret Tensor - Slim TF wrapper to compute LRP

The Layer-wise Relevance Propagation (LRP) algorithm explains a classifer's prediction specific to a given data point by attributing relevance scores to important components of the input by using the topology of the learned model itself.

This tensorflow wrapper provides simple and accessible stand-alone implementations of LRP for artificial neural networks.

Requirements

tensorflow >= 1.0.0
python >= 3
matplotlib >= 1.3.1
scikit-image > 0.11.3

Features

1. Model

This TF-wrapper considers the layers in the neural network to be in the form of a Sequence. A quick way to define a network would be

    net = Sequential([Linear(input_dim=784,output_dim=1296, act ='relu', batch_size=FLAGS.batch_size),
                 Linear(1296, act ='relu'), 
                 Linear(1296, act ='relu'),
                 Linear(10, act ='relu'),
                 Softmax()])

    output = net.forward(input_data)

2. Train the network

This net can then be used to propogate and optimize using

    trainer = net.fit(output, ground_truth, loss='softmax_crossentropy', optimizer='adam', opt_params=[FLAGS.learning_rate])

3. LRP - Layer-wise relevance propagation

And compute the contributions of the input pixels towards the decision by

    relevance = net.lrp(output, 'simple', 1.0)

the different lrp variants available are:

    'simple'and 'epsilon','flat','ww' and 'alphabeta' 

4. Compute relevances every layer backwards from the output to the input

Follow steps (1) from Features mentioned above.

   relevance_layerwise = []
   R = output
   for layer in net.modules[::-1]:
       R = net.lrp_layerwise(layer, R, 'simple')
       relevance_layerwise.append(R)

Examples

To run the given mnist examples,

    cd examples
    python mnist_linear.py --relevance=True

It downloads and extract the mnist datset, runs it on a neural netowrk and plots the relevances once the network is optimized. The relvances of the images can be viewed on the tensorboard using

    tensorboard --logdir=mnist_linear_logs

LRP for a pretrained model

Follow steps (1) and (3) from Features mentioned above.

The LRP Toolbox Paper

When using (any part) of this wrapper, please cite our paper

@article{JMLR:v17:15-618,
    author  = {Sebastian Lapuschkin and Alexander Binder and Gr{{\'e}}goire Montavon and Klaus-Robert M{{{\"u}}}ller and Wojciech Samek},
    title   = {The LRP Toolbox for Artificial Neural Networks},
    journal = {Journal of Machine Learning Research},
    year    = {2016},
    volume  = {17},
    number  = {114},
    pages   = {1-5},
    url     = {http://jmlr.org/papers/v17/15-618.html}
}

Misc

For further research and projects involving LRP, visit heatmapping.org

You can’t perform that action at this time.