Relevance aggregation for neural networks interpretability and knowledge discovery on tabular data

Welcome!

Relevance aggregation is a method for computing relevance scores for multilayer feed-forward neural networks trained on tabular data. It can be used to inspect the behavior of the trained models, identify biases, and for knowledge discovery. Besides the numerical values, the relevance scores can be visualized with table heatmaps and weighted t-SNE.

How to use

To configure all the hyperparameters of relevance aggregation, you only need to create a config.py file. An example can be downloaded here. It also contains the necessary documentation.

You will need Python 3 to run this code. Check if the needed libraries are installed with:

python3 check_dep.py

To train neural networks, use:

python3 train.py config.py

The relevance scores can then be computed by:

python3 get_relevances.py config.py

To generate the table heatmaps, run:

python3 heatsheets.py config.py

And for the weighted t-SNE visualization, run:

python3 visualize.py config.py

Data sets

You can download the datasets used in the experiments here.

Results

If you are looking for the trained Keras models and resulting table heatmaps from the main paper, you can find them here.

Libraries

This implementation of relevance aggregation uses the following Python 3.7 libraries:

• NumPy
• Pandas
• Matplotlib
• TensorFlow Keras
• SciPy
• Scikit-learn
• LRP Toolbox
• DeepLIFT

How to cite

If you use our code, methods, or results in your research, please consider citing the main publication of relevance aggregation:

• Bruno Iochins Grisci, Mathias J. Krause, Marcio Dorn. Relevance aggregation for neural networks interpretability and knowledge discovery on tabular data, Information Sciences, Volume 559, June 2021, Pages 111-129, DOI: 10.1016/j.ins.2021.01.052

Bibtex entry:

@article{grisci2021relevance,
  title={Relevance aggregation for neural networks interpretability and knowledge discovery on tabular data},
  author={Grisci, Bruno Iochins and Krause, Mathias J and Dorn, Marcio},
  journal={Information Sciences},
  year={2021},
  doi = {10.1016/j.ins.2021.01.052},
  publisher={Elsevier}
}

Contact information

• Bruno I. Grisci - PhD student (Institute of Informatics - UFRGS)

  • bigrisci@inf.ufrgs.br

• Dr. Mathias J. Krause - (Institute for Applied and Numerical Mathematics 2 - KIT)

  • mathias.krause@kit.edu

• Dr. Marcio Dorn - Associate Professor (Institute of Informatics - UFRGS)

  • mdorn@inf.ufrgs.br

• http://sbcb.inf.ufrgs.br/