Skip to content

fix: use parametrized test values in PrototypeRepresentationLearner tests#1626

Merged
taharallouche merged 7 commits intofairlearn:mainfrom
BALOGUN-DAVID:main
Feb 28, 2026
Merged

fix: use parametrized test values in PrototypeRepresentationLearner tests#1626
taharallouche merged 7 commits intofairlearn:mainfrom
BALOGUN-DAVID:main

Conversation

@BALOGUN-DAVID
Copy link
Contributor

[Improvement] – Fix parameterized unit tests and add multi-dimensional sensitive features coverage for PrototypeRepresentationLearner

Description

Context:
The existing unit tests for PrototypeRepresentationLearner in Fairlearn had a critical gap: while @pytest.mark.parametrize decorators were defined to test multiple input variations (including sensitive_features=None and pd.Series labels), these parameterized values were immediately overwritten inside the test functions. As a result, key code paths—like the None sensitive features and string-based label handling—were never tested. Additionally, no tests existed for multi-dimensional sensitive features (e.g., race + gender), which are essential for intersectional fairness assessment in real-world scenarios.

Approach:

  • Removed hardcoded overwrites in test_reconstruction and test_classification so pytest parametrize runs all intended cases.
  • Added two new test functions:
    1. test_statistical_parity_multi_dimension_sensitive_features
    2. test_transform_sensitive_features_with_two_dimensions
      These validate behavior with 2D sensitive features.
  • Simplified parametrize syntax for readability.
  • Imported _merge_columns utility to correctly merge multi-dimensional columns, mirroring internal learner behavior.

Impact:

  • Algorithmic Fairness: Properly tests both sensitive_features=None and multi-dimensional protected attributes.
  • Data Integrity: Ensures string-based labels (pd.Series) pass through the pipeline correctly.
  • Model Robustness: Multi-dimensional sensitive features now covered, guaranteeing the learner handles real-world intersectional groups.
  • Test Coverage: Increased from 2 to 6+ parametrize combinations, plus 2 new tests.

Visual Proof / Evidence

image

Screenshots/GIFs should show pytest results with all parametrize combinations passing and coverage report highlighting new multi-dimensional sensitive features tests.


Tests

  • New tests added
  • Existing tests adjusted (removed overwrites)

Unit Tests Summary:

  • test_reconstruction fixed
  • test_classification fixed
  • Added test_statistical_parity_multi_dimension_sensitive_features
  • Added test_transform_sensitive_features_with_two_dimensions


Copy link
Member

@taharallouche taharallouche left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM thank you @BALOGUN-DAVID 🙏 Before merging, can you please add a short entry in the change log for this bug fix ?

@BALOGUN-DAVID
Copy link
Contributor Author

Hi @taharallouche 👋 I've added the changelog entry, Please let me know if any changes are needed. Thanks!

@taharallouche
Copy link
Member

@BALOGUN-DAVID the change in this PR is the bug fix for degenerate sensitive feature values, not test parametrization though. Could you please correct the change log ?

@BALOGUN-DAVID
Copy link
Contributor Author

Done! I've updated the changelog entry. Thanks for the correction @taharallouche

@taharallouche taharallouche merged commit 53e022f into fairlearn:main Feb 28, 2026
38 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants