We present Aequitas, a directed fairness testing framework machine learning models. See the paper Automated Directed Fairness Testing for more details.
- Python 2.7.15
- numpy 1.14.5
- scipy 1.1.0
- scikit-learn 1.19.0
The authors used Pycharm CE 2017.2.3 as the development IDE.
There are 3 test generation strategies in our suite, namely Aequitas Random, Aequitas Semi-Directed and Aequitas Fully Directed. There are files to evaluate Fair SVM and Scikit-Learn classifiers trained on the same dataset.
The config file has the following data:
- params : The number of parameters in the data
- sensitive_param: The parameter under test.
- input_bounds: The bounds of each parameter
- classifier_name: Pickled scikit-learn classifier under test (only applicable to the sklearn files)
- threshold: Discrimination threshold.
- perturbation_unit: By what unit would the user like to perturb the input in the local search.
- retraining_inputs: Inputs to be used for the retraining. Please see this file.
python <filename>
eg. python Aequitas_Fully_Directed.py
- Please contact sakshi_udeshi@mymail.sutd.edu.sg for any comments/questions
If you use any part of this code, please cite the following paper
@inproceedings{aequitas,
title={Automated directed fairness testing},
author={Udeshi, Sakshi and Arora, Pryanshu and Chattopadhyay, Sudipta},
booktitle={Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering},
pages={98--108},
year={2018}
}