Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature/metrics #21

Merged
merged 32 commits into from
May 5, 2021
Merged

feature/metrics #21

merged 32 commits into from
May 5, 2021

Conversation

Hilly12
Copy link
Contributor

@Hilly12 Hilly12 commented Apr 29, 2021

  • Bias package with metrics; add, refactor metrics class imbalance, emd, ks distance, kl divergence, js divergence, lp norm
  • Visualization tools added
  • Utils for binning, type inference, distribution inference added
  • Redundant hierarchy from documentation removed and minor changes to docs
  • Compas dataset added

#9 #17 #18 #25

@Hilly12 Hilly12 marked this pull request as ready for review April 29, 2021 16:33

xx, yy = np.meshgrid(space, space)
distance_space = np.abs(xx - yy)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if we need to normalize the distance space

src/fairlens/bias/metrics.py Outdated Show resolved Hide resolved
g_counts = df[pred].groupby(target_attr)[target_attr].aggregate(Count="count")["Count"].to_dict()

plt.plot(g_counts.keys(), g_counts.values())

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At the moment the curves plot the raw data. It may make more sense to bin before drawing the curves.


from . import utils


Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be worth adding a 'normalize' keyword which plots the pdf instead of the target attr distribution, or maybe its worth having a different method to plot pdfs.

.github/workflows/ci.yml Show resolved Hide resolved
docs/api-docs/bias.rst Show resolved Hide resolved
docs/conf.py Show resolved Hide resolved
@@ -0,0 +1,112 @@
from typing import Any, Dict, List

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might have been good to separate this work into another PR, that way we keep the goals of each PR isolated to a single issue.

src/fairlens/bias/utils.py Outdated Show resolved Hide resolved
src/fairlens/bias/utils.py Outdated Show resolved Hide resolved
Copy link

@jamied157 jamied157 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks for making the changes!

@Hilly12 Hilly12 merged commit a11b4e9 into main May 5, 2021
@Hilly12 Hilly12 deleted the feature/metrics branch May 6, 2021 10:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants