On Fairness and Calibration

Link zum Paper

Summary

• A noteworthy framework to quantify fairness in classification is Equalized Odds which constrains classification algorithms such that no error type (false-positive or false-negative) disproportionately affects any population subgroup.

• Calibration: Set of people who receive a predicted probability of p, we would like a p fraction of the members of this set to be positive instances of the classification problem.

• Fairness between two groups: calibration condition holds simultaneously for the set within each of these groups as well

• Build two classifiers for the selected protected groups and compare them

• Both classifiers should be calibrated with respect to groups G1(protected) and G2(not protected) to prevent discrimination

• Proven that a classifier cannot achieve both calibration and Equalized Odds, even in an approximate sense, except in the most trivial of cases.

• Find relaxation that seeks to satisfy a single equal-cost constraint while maintaining calibration for each group Gt.

• Build cost function f_t thats a linear combination of the classifiers false- positvies and negatives rate with arbitrary dependence on the group’s base rate µ

• Example for implementation of realxed equalized odds for logistic regression

• Examples for real world applications. Very unsettling.

Tasks and Ideas

• Check if dependency between calibration and equal odds can be found (advanced)
• Implementation of relaxed Equalized odds for LVQ