Code for paper titled
On the Long-term Impact of Algorithmic Policies: Effort Unfairness and Feature Segregation through the Lens of Social Learning to appear at ICML 2019
- Vedant Nanda, MPI-SWS
- Hoda Heidari, ETH Zürich
- Krishna P. Gummadi, MPI-SWS
- python 3.5.3 or above
Libraries needed to run the code:
To install all these, cd to effort_reward_fairness and execute
pip install -r requirements.txt. You might need sudo permission for installation.
Reproduction of experiments and results in the paper
Once dependencies have been installed, cd to the
We first need to generate explanations for users in both the train and the test set. To do so, run
python experiment.py trainand
python experiment.py test. These can be run either simultaneously or sequentially (since they generate explanations for different parts of the dataset). However, you must run both these commands before proceeding to the next step. As a consequence of running
python experiment.py testthe plot containing analysis of different disparity measure for different models is generated and stored in
./effort_reward_fairness/results/StudentPerf/disparity_plots/all_disp_in_one_test.pdf. This is the plot in Fig 3 in the paper.
To generate plot in Fig 1 and 2, run
python effort_reward_function_plots.py train. This will generate Average Reward as a function of effort and Average effort as a function of reward. You will find
Reward_vs_Average_Effort_together.pdfin the directory
./effort_reward_fairness/results/StudentPerf/disparity_plots/. These can also be seen in Fig 1 and 2 in the paper.
Once you have completed step 1, then run
python long_term_impact.py. This will generate new set of feature vectors which correspond to action taken by users on explanations given in step 1.
After step 3 completes, execute
python utility_thresholds.pyto see the effect of algorithmic policies on segregation. As a result of executing
python utility_thresholds.pyyou will find
segregation_ACI.pdfin the directory
./effort_reward_fairness/results/StudentPerf/segregation_plots. These are included in Fig 4 in the paper.
Now to generate plots for models with Fairness Constraints, open experiment.py and change the global variable
FAIRNESS_CONSTRAINTSto True. Save the file. Now repeat steps 1, 3 and 4 listed above (step 2 is optional, results of step 2 for models with Fairness Constraints are not included in the paper). **
Once you have done step 5, you'll find
segregation_ACI_fc.pdfin the directory
./effort_reward_fairness/results/StudentPerf/segregation_plots. Thes plots are included in Fig 5 in the paper.
Code for preprocessing the Student Performance Dataset can be found in
./util/datasets/data/student_performance/preprocessing.ipynb. An executable python file (
preprocessing.py) of this notebook is also available in the same directory.
** To run Fairness Constraints you need the file
trained_linregfc_StudentPerf.mat in the
./effort_reward_fairness directory. For easy reproduction of results we have included this file, however this is generated using MATLAB code in the directory
./Fairness_constraints and can be found in the directory
./Fairness_constraints/Output after you run
Caution: This is a work in progress, some TODOs (particularly in group_explanations.py need to be carefully looked at).