Fairness Metrics

Fairness Metrics

Evaluate model bias with respect to protected attributes (gender, race, age, etc.).

Metrics

Metric	Definition	Fair if
Demographic Parity	P(Ŷ=1|A=a) equal across groups	ratio ≥ 0.8
Disparate Impact	min selection rate / max selection rate	ratio ≥ 0.8 (4/5 rule)
Equal Opportunity	TPR equal across groups	difference < 0.1
Equalized Odds	TPR + FPR equal across groups	both differences < 0.1

Python API

from scomp_link import FairnessMetrics

fm = FairnessMetrics(
    y_true=y_test,
    y_pred=predictions,
    sensitive_feature=df['gender']
)

# Compute all metrics
report = fm.compute_all()

# Summary table
print(fm.summary(report))

# Visualization
fig = fm.plot_fairness_report(report)

# Individual metrics
dp = fm.demographic_parity()
di = fm.disparate_impact()
eo = fm.equalized_odds()
eop = fm.equal_opportunity()

CLI

scomp-link fairness --data predictions.csv --target y_true --predicted y_pred --sensitive gender

Interpreting Results

• 4/5 Rule (Disparate Impact): If the selection rate for any group is less than 80% of the highest group's rate, there may be adverse impact. This is the most commonly used legal standard.
• Equalized Odds: The strictest criterion — requires both TPR and FPR to be equal across groups.