Skip to content

Pacmed/sensible-local-interpretations

master
Switch branches/tags

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
 
 
sli
 
 
 
 
 
 
 
 
 
 
 
 

sensible local interpretations

Note: this repo is actively maintained. For any questions please file an issue.

This project aims to provide a way to interpret individual predictions made by a model in terms of 3 things: (1) uncertainty, (2) contribution, and (3) sensitivity. The methods here are model-agnostic and fast. The main source code is in the sli folder.

definitions

  1. uncertainty
    • in addition to original model, train model with one class upweighted, one class downweighted
    • use these additional models to get info about uncertainty (mostly aleatoric)
    • can define uncertainty as overconfident prediction - underconfident prediction
  2. ICE-sensitivity
    • how does changing this feature change the prediction, holding all other features constant?
  3. ICE-contribution
    • how does this prediction differ from a typical prediction?

usage

The outcome allows for an interactive exploration of how a model makes its prediction: demo.

Install with pip install git+https://github.com/Pacmed/sensible-local-interpretations

Let

X : ndarray
    Training data, used to properly sample the curves for the interpretation
feature_names: list[str]
            Feature names, only used for plotting, returning tables

Given one trained model and explaining one instance:

from sli import Explainer
explainer = Explainer(X)
expl_dict = explainer.explain_instance(x, model.predict_proba, return_table=False)
explainer.viz_expl(expl_dict, filename='out.html')

Given a list of three models (with the best model in the middel of the list):

from sli import Explainer
explainer = Explainer(X_train, feature_names=feature_names)
explainer.calc_percentiles(models[0].predict_proba, models[1].predict_proba, 
                           models[2].predict_proba)
expl_dicts = [expl_dicts.append(explainer.explain_instance(x, models[i].predict_proba, 
              class_num, return_table=False)) for i in range(3)]
explainer.viz_expl(expl_dicts[1], [expl_dicts[0], expl_dicts[2]], 
                   show_stds=True, # display stddevs for the feature importances
                   point_id='', # display id of this point
                   filename='out.html') # saves to fully-contained interactive html file

Releases

No releases published

Packages

No packages published

Contributors 4

  •  
  •  
  •  
  •