ENH add mitigation algorithm "Mechanisms for Fair Classification" by Zafar et al.

[hildeweerts]

The paper Fairness Constraints: Mechanisms for Fair Classification by Zafar et al. introduces a constrained learning algorithm for demographic parity. The goal of this task is to implement the logistic regression algorithm described in this paper in Fairlearn.

• An existing Python implementation of the authors can be found here.
• The paper describes two different formulations: (1) maximizing accuracy under fairness constraints, and (2) maximizing fairness under accuracy constraints. We can choose one of them (I think (1) is most natural) or implement both.
• I would suggest to design the API similar to sklearn.linear_model.LogisticRegression.

Completing this item requires:

• technique code in fairlearn.linear_model (this is my suggestion following scikit-learn, but the specific naming is up for discussion).
• unit tests in test.unit.linear_model
• descriptive API reference (directly in the docstring)
• a short user guide in docs.user_guide.mitigation.rst

A fully fledged example notebook is not required.

To claim this task please respond below. Of course, you can also ask questions!

Closely related to #1027

[rensoostenbach]

I will start on this issue next week :). While working on this, should I in some way keep in mind that this is closely related to #1027, or is it just meant as extra information?

[hildeweerts]

Awesome @rensoostenbach! In case somebody else starts working on #1027 it would be good to coordinate, but for now it is just additional info.

[rensoostenbach]

My plan to approach this is as follows:
I will create a FairLogisticRegression class in fairlearn.linear_model._fair_logistic.py (exact naming to be determined). This class inherits from the sklearn LogisticRegression and will be changed in the places where adaptations need to be made (similarly to how imbalanced-learn does it for a BalancedRandomForestClassifier).
The main difference between the sklearn LogisticRegression and the one proposed in the paper, is that the minimization function has some extra constraints regarding the fairness, hence why I think it will be useful to only change the code in those places.
These adaptations are as follows:

• Adding a constraints parameter to the optimize.minimize function (here in sklearn, as is done in the code of the paper)
• Making sure that the FairLogisticRegression can handle sensitive attributes. Would this be as easy as dropping them from X such that the model does not use them for training, or is there something else I should be aware of?

The plan is to start with only the lbfgs solver, since it uses the same optimize.minimize function in sklearn as well as the code of the paper. Also, it is the default solver for LogisticRegression.

I would like to get some feedback on this approach, and perhaps some things I should take caution of when inheriting from LogisticRegression, etc. Thanks!

[adrinjalali]

sklearn has a new GLM framework with a few models implemented under it, and it'd be nice to see if you could extend those instead of implementing from scratch. An example is TweedieRegressor: https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.TweedieRegressor.html

[rensoostenbach]

sklearn has a new GLM framework with a few models implemented under it, and it'd be nice to see if you could extend those instead of implementing from scratch. An example is TweedieRegressor: https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.TweedieRegressor.html

This seems to be relatively doable! From the source code, the same optimize.minimize function is used, and the TweedieRegressor only uses the lbfgs solver, making it quite easy to extend it to.