Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Normalize cost-benefit matrix values #1099

Merged
merged 7 commits into from
Aug 25, 2020
Merged

Normalize cost-benefit matrix values #1099

merged 7 commits into from
Aug 25, 2020

Conversation

angela97lin
Copy link
Contributor

Closes #1098

@angela97lin angela97lin added this to the August 2020 milestone Aug 25, 2020
@angela97lin angela97lin self-assigned this Aug 25, 2020
@codecov
Copy link

codecov bot commented Aug 25, 2020

Codecov Report

Merging #1099 into main will not change coverage.
The diff coverage is 100.00%.

Impacted file tree graph

@@           Coverage Diff           @@
##             main    #1099   +/-   ##
=======================================
  Coverage   99.91%   99.91%           
=======================================
  Files         192      192           
  Lines       10719    10719           
=======================================
  Hits        10710    10710           
  Misses          9        9           
Impacted Files Coverage Δ
evalml/objectives/cost_benefit_matrix.py 100.00% <100.00%> (ø)
.../tests/objective_tests/test_cost_benefit_matrix.py 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 554848c...817cae4. Read the comment docs.

@angela97lin angela97lin marked this pull request as ready for review August 25, 2020 14:49
Copy link
Collaborator

@jeremyliweishih jeremyliweishih left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🦺

Copy link
Contributor

@dsherry dsherry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, good call on this! I'd expect no change in ML performance from this, but its a good clarification. The benefit: if a user runs the same cost-benefit on two different training sets, where one has more rows than the other, the cost-benefit scores will have the same bounds.

I left one request for a math clarification.

evalml/objectives/cost_benefit_matrix.py Outdated Show resolved Hide resolved
Copy link
Contributor

@kmax12 kmax12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think we need to update the docs after this change

image

@angela97lin
Copy link
Contributor Author

@kmax12 Thanks for reminding me, updated docs here: https://evalml.alteryx.com/en/1098_normalize/demos/cost_benefit_matrix.html

@angela97lin angela97lin merged commit 30cc109 into main Aug 25, 2020
@angela97lin angela97lin deleted the 1098_normalize branch August 25, 2020 16:46
@dsherry dsherry mentioned this pull request Aug 25, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Normalize cost-benefit matrix values
4 participants