Skip to content

Updated Objectives API to allow for sample weighting - #2433

Merged
christopherbunn merged 4 commits into
mainfrom
1867_objective_weighting
Jun 25, 2021
Merged

Updated Objectives API to allow for sample weighting#2433
christopherbunn merged 4 commits into
mainfrom
1867_objective_weighting

Conversation

@christopherbunn

Copy link
Copy Markdown
Contributor

Added the sample_weight parameter to objective_function and score in ObjectiveBase. Also updated most scikit-learn based standard metrics to utilize sample weights where appropriate.

Resolves #1867

@codecov

codecov Bot commented Jun 23, 2021

Copy link
Copy Markdown

Codecov Report

Merging #2433 (eb10dbe) into main (209d5e1) will increase coverage by 0.1%.
The diff coverage is 100.0%.

Impacted file tree graph

@@           Coverage Diff           @@
##            main   #2433     +/-   ##
=======================================
+ Coverage   99.7%   99.7%   +0.1%     
=======================================
  Files        283     283             
  Lines      25478   25500     +22     
=======================================
+ Hits       25378   25400     +22     
  Misses       100     100             
Impacted Files Coverage Δ
evalml/objectives/cost_benefit_matrix.py 100.0% <100.0%> (ø)
evalml/objectives/fraud_cost.py 100.0% <100.0%> (ø)
evalml/objectives/lead_scoring.py 100.0% <100.0%> (ø)
evalml/objectives/objective_base.py 100.0% <100.0%> (ø)
evalml/objectives/standard_metrics.py 100.0% <100.0%> (ø)
...lml/tests/objective_tests/test_standard_metrics.py 100.0% <100.0%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 209d5e1...eb10dbe. Read the comment docs.

@christopherbunn
christopherbunn force-pushed the 1867_objective_weighting branch 2 times, most recently from ec5469e to fc9e816 Compare June 25, 2021 03:36
@christopherbunn
christopherbunn marked this pull request as ready for review June 25, 2021 13:53

@jeremyliweishih jeremyliweishih left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@freddyaboulton freddyaboulton left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@christopherbunn Looks good to me! Is there a plan to use this in automl? Just curious

self.false_negative = false_negative

def objective_function(self, y_true, y_predicted, X=None):
def objective_function(self, y_true, y_predicted, X=None, sample_weight=None):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you update the docstring here and in fraud and lead scoring?

@christopherbunn
christopherbunn force-pushed the 1867_objective_weighting branch from fc9e816 to eb10dbe Compare June 25, 2021 16:33
@christopherbunn
christopherbunn merged commit 020e434 into main Jun 25, 2021
@dsherry dsherry mentioned this pull request Jul 2, 2021
@freddyaboulton
freddyaboulton deleted the 1867_objective_weighting branch May 13, 2022 15:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Objectives: support weighting

4 participants