This artifact includes source code, datasets and models of the paper "Efficient White-Box Fairness Testing through Gradient Search" by Lingfeng Zhang, Yueling Zhang, Min Zhang in the proceedings of ISSTA'21.
Conference Website: https://conf.researchr.org/details/issta-2021/issta-2021-technical-papers/34/Efficient-White-Box-Fairness-Testing-through-Gradient-Search
Paper: https://dl.acm.org/doi/10.1145/3460319.3464820
1.1 Environment
Python3.6 + Tensorflow2.0 or later versions are recommended. The following packages are also required: numpy, scikit-learn, pandas, joblib.
1.2 A Small Example
We provide a small example to validate the general functionality by generating some individual discriminatory instances with ADF, EIDIG-5, and EIDIG-INF, respectively. In the corresponding file directory, enter in the terminal 'python test.py'. You will get statistical results in the terminal and have discriminatory instances saved to 'logging_data/logging_data_from_tests/complete_comparison/'. Less than 30 min will be taken for a personal computer.
2.1 Effectiveness/Efficiency Validation
We reproduce the main experiments in 'reproducing.py'. This module compares the effectiveness and efficiency between ADF and EIDIG in the settings adopted in our paper. You can run it to validate our claims. It will take a few days for a personal computer, or you can run different parts of experiments in parallel for acceleration. The statistical results will be immediately reported in the terminal, and the data for further analysis are saved to 'logging_data/logging_data_from_tests/'.
2.2 Model Evaluation
In the 'model_evaluation' folder, you can evaluate classification performance and discrimination degree on the models provided by us. Alternatively, you can retrain the models by yourself with our data, or with the data generated by 'reproducing.py', and then do the evaluation. The statistical results will also be immediately reported in the terminal.