No description, website, or topics provided.
Jupyter Notebook Python
Switch branches/tags
Nothing to show
Clone or download
Latest commit 578bc45 Aug 23, 2017
Permalink
Failed to load latest commit information.
inception First commit Jul 10, 2017
influence First commit Jul 10, 2017
scripts First commit Jul 10, 2017
LICENSE.md Create LICENSE.md Jul 10, 2017
README.md Update README.md Aug 23, 2017
__init__.py First commit Jul 10, 2017

README.md

Understanding Black-box Predictions via Influence Functions

This code replicates the experiments from the following paper:

Pang Wei Koh and Percy Liang

Understanding Black-box Predictions via Influence Functions

International Conference on Machine Learning (ICML), 2017.

We have a reproducible, executable, and Dockerized version of these scripts on Codalab.

The datasets for the experiments can also be found at the Codalab link.

Dependencies:

  • Numpy/Scipy/Scikit-learn/Pandas
  • Tensorflow (tested on v1.1.0)
  • Keras (tested on v2.0.4)
  • Spacy (tested on v1.8.2)
  • h5py (tested on v2.7.0)
  • Matplotlib/Seaborn (for visualizations)

A Dockerfile with these dependencies can be found here: https://hub.docker.com/r/pangwei/tf1.1/


In this paper, we use influence functions --- a classic technique from robust statistics --- to trace a model's prediction through the learning algorithm and back to its training data, thereby identifying training points most responsible for a given prediction. To scale up influence functions to modern machine learning settings, we develop a simple, efficient implementation that requires only oracle access to gradients and Hessian-vector products. We show that even on non-convex and non-differentiable models where the theory breaks down, approximations to influence functions can still provide valuable information. On linear models and convolutional neural networks, we demonstrate that influence functions are useful for multiple purposes: understanding model behavior, debugging models, detecting dataset errors, and even creating visually-indistinguishable training-set attacks.

If you have questions, please contact Pang Wei Koh (pangwei@cs.stanford.edu).