Skip to content
FairVis: Visual Analytics for Discovering Intersectional Bias in Machine Learning
JavaScript Jupyter Notebook CSS Python HTML
Branch: master
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
models
public
src
.gitignore
LICENSE.md
README.md
package-lock.json
package.json
preprocess.py
teaser.png

README.md

FairVis: Visual Analytics for Discovering Intersectional Bias in Machine Learning

FairVis is a visual analytics system that allows users to audit their classification models for intersectional bias. Users can generate subgroups of their data and investigate if a model is underperforming for certain populations.

FairVis: Visual Analytics for Discovering Intersectional Bias in Machine Learning
Ángel Alexander Cabrera, Will Epperson, Fred Hohman, Minsuk Kahng, Jamie Morgenstern, Duen Horng (Polo) Chau IEEE Conference on Visual Analytics Science and Technology (VAST). 2019.

teaser figure

In this example we show how FairVis can be used on the COMPAS dataset to find significant disparities in false positive rates between African American and Caucasian defendants that are not supported by base rates. The primary components of the system are the following:

A. View distributions of the dataset's features and generate subgroups.

B. Visualize subgroup performance in relation to selected metrics.

C. Compare selected subgroups and view details.

D. Find suggested underperforming subgroups and similar groups.

The full paper, currently under review, can be found here, detailing the system and use cases.

Installation

Clone the repository:

git clone https://github.com/poloclub/FairVis.git

Then initialize the React project by running

npm install

Usage

Run the server with

npm start

Adding a new dataset

  1. Run a model on your data and and create a new file with the last two columns being the output class (between 0-1) of the model and the ground truth labels (0 or 1). Note that only binary classification is currently supported. Examples of models in Jupyter Notebook format can be found in ./models.

  2. Run the preprocess.py script on your classified data, e.g. python3 preprocess.py my-data-with-classes.csv. Additional options for the helper function can be found using python3 preprocess.py -h.

  3. Save the processed file to ./src/data/.

  4. Import the file in the src/components/Welcome.js component.

  5. Add a new row to the table in Welcome.js around line 140 in the form of the other datsets.

Researchers

Name Affiliation
Ángel Alexander Cabrera Georgia Tech
Will Epperson Georgia Tech
Fred Hohman Georgia Tech
Minsuk Kahng Georgia Tech
Jamie Morgenstern Georgia Tech
Duen Horng (Polo) Chau Georgia Tech

Citation

@article{cabrera2019fairvis,
  title={FairVis: Visual Analytics for Discovering Intersectional Bias in Machine Learning},
  author={Cabrera, {'A}ngel Alexander and Epperson, Will and Hohman, Fred and Kahng, Minsuk and Morgenstern, Jamie and Chau, Duen Horng},
  journal={IEEE Conference on Visual Analytics Science and Technology (VAST)},
  year={2019},
  publisher={IEEE}
  url={https://cabreraalex.com/#/paper/fairvis}
}

License

MIT License. See LICENSE.md.

You can’t perform that action at this time.