Probability Calibration on Fairlearn reduction methods

[ankitaash]

We need to perform post processing techniques (sigmoid/isotonic) on fairlearn reduction methods such as Exponentiated Gradient and Grid Search.
However Exponentiated Gradient creates multiple models and assigns randomised predictions based on attributes weights__
How can we select one model to apply post processing on or can it be applied on the complete stacked models at once.

[romanlutz]

You can access them via predictors_. Note that they may not satisfy the fairness constraints individually.

Were you looking for guidance on which one to select?

[ankitaash]

You can access them via predictors_. Note that they may not satisfy the fairness constraints individually.

Were you looking for guidance on which one to select?

Hi @romanlutz,
yes, as written in the documentation of predict method -The prediction on each data point in X is obtained by first picking a random predictor according to the probabilities in weights and then applying it. Different predictors can be chosen on different data points._

How do we select one predictor to perform post processing calibration?

[romanlutz]

Using expgrad.predictors_[i] for whichever index you like. It's just an array. I recommend examining their fairness-related stats using MetricFrame, and again: they may not perform as well as ExponentiatedGradient because that's the point of applying the randomization.