Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updated Objectives API to allow for sample weighting #2433

Merged
merged 4 commits into from
Jun 25, 2021

Conversation

christopherbunn
Copy link
Contributor

Added the sample_weight parameter to objective_function and score in ObjectiveBase. Also updated most scikit-learn based standard metrics to utilize sample weights where appropriate.

Resolves #1867

@codecov
Copy link

codecov bot commented Jun 23, 2021

Codecov Report

Merging #2433 (eb10dbe) into main (209d5e1) will increase coverage by 0.1%.
The diff coverage is 100.0%.

Impacted file tree graph

@@           Coverage Diff           @@
##            main   #2433     +/-   ##
=======================================
+ Coverage   99.7%   99.7%   +0.1%     
=======================================
  Files        283     283             
  Lines      25478   25500     +22     
=======================================
+ Hits       25378   25400     +22     
  Misses       100     100             
Impacted Files Coverage Δ
evalml/objectives/cost_benefit_matrix.py 100.0% <100.0%> (ø)
evalml/objectives/fraud_cost.py 100.0% <100.0%> (ø)
evalml/objectives/lead_scoring.py 100.0% <100.0%> (ø)
evalml/objectives/objective_base.py 100.0% <100.0%> (ø)
evalml/objectives/standard_metrics.py 100.0% <100.0%> (ø)
...lml/tests/objective_tests/test_standard_metrics.py 100.0% <100.0%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 209d5e1...eb10dbe. Read the comment docs.

@christopherbunn christopherbunn force-pushed the 1867_objective_weighting branch 2 times, most recently from ec5469e to fc9e816 Compare June 25, 2021 03:36
@christopherbunn christopherbunn marked this pull request as ready for review June 25, 2021 13:53
Copy link
Collaborator

@jeremyliweishih jeremyliweishih left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Copy link
Contributor

@freddyaboulton freddyaboulton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@christopherbunn Looks good to me! Is there a plan to use this in automl? Just curious

@@ -35,7 +35,7 @@ def __init__(self, true_positive, true_negative, false_positive, false_negative)
self.false_positive = false_positive
self.false_negative = false_negative

def objective_function(self, y_true, y_predicted, X=None):
def objective_function(self, y_true, y_predicted, X=None, sample_weight=None):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you update the docstring here and in fraud and lead scoring?

@christopherbunn christopherbunn merged commit 020e434 into main Jun 25, 2021
@dsherry dsherry mentioned this pull request Jul 2, 2021
@freddyaboulton freddyaboulton deleted the 1867_objective_weighting branch May 13, 2022 15:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Objectives: support weighting
4 participants