Skip to content
Comparing fairness-aware machine learning techniques.
Branch: master
Clone or download
Carlos Scheidegger
Latest commit 97bca16 Jan 28, 2019
Type Name Latest commit message Commit time
Failed to load latest commit information.
.gitignore Kamishima code runs.. Jan 8, 2018
.gitmodules Added reproduce and got rid of extraneous files Jun 30, 2017
AUTHORS.txt license Feb 12, 2018
Makefile pip install updates to the readme Jul 18, 2018 making this pip installable Jun 14, 2018 created a changelog Jan 27, 2019
pip_update_commands.txt commands to update the pip version Jun 14, 2018
requirements.txt Merge branch 'master' of… Feb 10, 2018 bugfix for pathlib on preprocess Jan 28, 2019

This repository is meant to facilitate the benchmarking of fairness aware machine learning algorithms.

The associated paper is:

A comparative study of fairness-enhancing interventions in machine learning by Sorelle A. Friedler, Carlos Scheidegger, Suresh Venkatasubramanian, Sonam Choudhary, Evan P. Hamilton, and Derek Roth.

To install this software run:

$ pip3 install fairness

The below instructions are still in the process of being updated to work with the new pip install-able version.

To run the benchmarks:

$ from fairness.benchmark import run
$ run()

This will write out metrics for each dataset to the results/ directory.

To generate graphs and other analysis run:

$ python3

If you do not yet have all the packages installed, you may need to run:

$ pip install -r requirements.txt

Optional: The benchmarks rely on preprocessed versions of the datasets that have been included in the repository. If you would like to regenerate this preprocessing, run the below command before running the benchmark script:

$ python3

To add new datasets or algorithms, see the instructions in the readme files in those directories.

OS-specific things

On Ubuntu

(We tested on Ubuntu 16.04, your mileage may vary)

You'll need python3-dev:

$ sudo apt-get install python3-dev

Additional analysis-specific requirements

To regenerate figures (this is messy right now. we're working on it)

Python requirements (use pip):

  • ggplot

System requirements:

R package requirements (use install.packages):

  • rmarkdown
  • stringr
  • ggplot2
  • dplyr
  • magrittr
  • corrplot
  • robust
You can’t perform that action at this time.