Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CategoryAdherence metric #462

Closed
frances-h opened this issue Oct 9, 2023 · 0 comments · Fixed by #475 or #499
Closed

Add CategoryAdherence metric #462

frances-h opened this issue Oct 9, 2023 · 0 comments · Fixed by #475 or #499
Assignees
Labels
feature request Request for a new feature
Milestone

Comments

@frances-h
Copy link
Contributor

frances-h commented Oct 9, 2023

Problem Description

As a user, I would like a metric that verifies I do not have invalid discrete values.

Expected behavior

  • Add a new single_column metric that calculates the percent of values that match at least 1 value from the real data.
  • This metric takes in categorical and boolean sdtype columns.

Attributes

The metric should have the following attributes:

  • name: 'CategoryAdherence'
  • goal: Goal.MAXIMIZE
  • min_value: 0.0
  • max_value: 1.0

Methods

The metric should also define the following methods

  • compute(real_data, synthetic_data): Compute the score for the metric. The returned score should be the percent of synthetic values that match at least 1 value in the real data. Null values should be counted as a separate category.
    • Parameters:
      • (required) real_data: A pandas.Series object with the column of real data
      • (required) synthetic_data: A pandas.Series object with the column of synthetic data
    • Returns: The score for this metric
>>> from sdmetrics.single_column import CategoryAdherence
>>> CategoryAdherence.compute(
	real_data=real_table['ethnicity'],
	synthetic_data=synthetic_table['ethnicity'])
1.0
>>> CategoryAdherence.compute_breakdown(
	real_data=real_table['ethnicity'],
	synthetic_data=synthetic_table['ethnicity'])
{ 'score': 1.0 }
@frances-h frances-h added feature request Request for a new feature new Label applied to new issues labels Oct 9, 2023
@amontanez24 amontanez24 removed the new Label applied to new issues label Oct 23, 2023
@amontanez24 amontanez24 added this to the 0.13.0 milestone Oct 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature request Request for a new feature
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants