Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor make_scorer for group fairness metrics #174

Merged
merged 2 commits into from
May 11, 2020

Conversation

chkoar
Copy link
Contributor

@chkoar chkoar commented May 6, 2020

No description provided.

Copy link
Collaborator

@hoffmansc hoffmansc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like it!

Could you also adjust the docs? Just replace the two lines with make_scorer.

aif360/sklearn/metrics/metrics.py Outdated Show resolved Hide resolved
tests/sklearn/test_metrics.py Outdated Show resolved Hide resolved
tests/sklearn/test_metrics.py Outdated Show resolved Hide resolved
@chkoar
Copy link
Contributor Author

chkoar commented May 8, 2020

Hey @hoffmansc, I think that addressed your comments. Is there something else?

Apart from that I think that the default scikit-learn scorers could be patched (probably using a patch function) so one could run:

cross_val_score(model, X, y, scoring="statistical_parity_difference")

or getting the scorer from sklearn.metrics.get_scorer just passing the string:

from sklearn.metrics import get_scorer
scorer = get_scorer("statistical_parity_difference")

Monkey-patching might be a controversial thing in the community.
If this is not the case, I could add this functionality in this or another PR.

Copy link
Collaborator

@hoffmansc hoffmansc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@hoffmansc hoffmansc merged commit bc55353 into Trusted-AI:master May 11, 2020
@hoffmansc
Copy link
Collaborator

hoffmansc commented May 11, 2020

@chkoar I don't feel that strongly about it but I do think it might add a little bit of confusion about what functionality is coming from which library... For now, the current syntax doesn't seem too cumbersome to me

@chkoar
Copy link
Contributor Author

chkoar commented Jun 2, 2020

but I do think it might add a little bit of confusion about what functionality is coming from which library

This might be true. In any case I opened an issue in the scikit-learn's upstream repo.

Illia-Kryvoviaz pushed a commit to Illia-Kryvoviaz/AIF360 that referenced this pull request Jun 7, 2023
* Refactor make_scorer for group fairness metrics

* Address hoffmansc comments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants