Skip to content

Make get_prediction_vs_actual_data() a public method #1553

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Dec 15, 2020
Merged

Make get_prediction_vs_actual_data() a public method #1553

merged 3 commits into from
Dec 15, 2020

Conversation

christopherbunn
Copy link
Contributor

Made get_prediction_vs_actual_data() a public method, added a check for outlier_threshold, and added it to API documentation.

Resolves #1542

@christopherbunn christopherbunn force-pushed the 1542_get_prediction_vs_actual_data branch from 2a23252 to 9969dd0 Compare December 14, 2020 19:29
@codecov
Copy link

codecov bot commented Dec 14, 2020 •
edited
Loading

Codecov Report

Merging #1553 (8bf755d) into main (61e1a62) will increase coverage by 0.1%.
The diff coverage is 100.0%.

Impacted file tree graph

@@            Coverage Diff            @@
##             main    #1553     +/-   ##
=========================================
+ Coverage   100.0%   100.0%   +0.1%     
=========================================
  Files         234      234             
  Lines       16803    16827     +24     
=========================================
+ Hits        16795    16819     +24     
  Misses          8        8
Impacted Files Coverage Δ
evalml/model_understanding/__init__.py 100.0% <ø> (ø)
evalml/model_understanding/graphs.py 99.7% <100.0%> (+0.1%) ⬆️
...lml/tests/model_understanding_tests/test_graphs.py 100.0% <100.0%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 61e1a62...8bf755d. Read the comment docs.

@christopherbunn christopherbunn marked this pull request as ready for review December 14, 2020 19:47
freddyaboulton
freddyaboulton approved these changes Dec 14, 2020
View reviewed changes
Copy link
Contributor

@freddyaboulton freddyaboulton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@christopherbunn I think this is good to merge!

evalml/model_understanding/graphs.py
def _get_prediction_vs_actual_data(y_true, y_pred, outlier_threshold):
"""Helper method to help calculate the y_true and y_pred dataframe, with a column for outliers"""
def get_prediction_vs_actual_data(y_true, y_pred, outlier_threshold=None):
"""Combines y_true and y_pred into a single dataframe and adds a column for outliers. Used in `graph_prediction_vs_actual()`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice docstring!

angela97lin
angela97lin reviewed Dec 14, 2020
View reviewed changes
evalml/model_understanding/graphs.py Outdated
Comment on lines 514 to 515
y_true (pd.Series): The real target values of the data
y_pred (pd.Series): The predicted values outputted by the regression model.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like from impl that woodwork is supported too? Could you update docstr / tests? Otherwise I can wait for this to be merged and add it to a PR of mine too hehe

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep I can add it to this PR!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sweet, thank you!!

angela97lin
angela97lin approved these changes Dec 14, 2020
View reviewed changes
Copy link
Contributor

@angela97lin angela97lin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!! Thanks for this 🥳

@christopherbunn
Copy link
Contributor Author

Looks like one of the windows checks threw an internal pytest error. Since the other two windows checks passed and the particular failed test passed after rerunning I think it's a one-off error.

@christopherbunn christopherbunn force-pushed the 1542_get_prediction_vs_actual_data branch from c46fd27 to dd887eb Compare December 14, 2020 22:00
@christopherbunn christopherbunn force-pushed the 1542_get_prediction_vs_actual_data branch from dd887eb to 8bf755d Compare December 15, 2020 14:26
@christopherbunn christopherbunn merged commit 39ce817 into main Dec 15, 2020
@christopherbunn christopherbunn deleted the 1542_get_prediction_vs_actual_data branch December 15, 2020 14:50
dsherry
dsherry reviewed Dec 15, 2020
View reviewed changes
Copy link
Contributor

@dsherry dsherry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@christopherbunn this is excellent! 🚀

@dsherry dsherry mentioned this pull request Dec 29, 2020
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dsherry dsherry dsherry left review comments

@angela97lin angela97lin angela97lin approved these changes

@freddyaboulton freddyaboulton freddyaboulton approved these changes

@bchen1116 bchen1116 Awaiting requested review from bchen1116

@eccabay eccabay Awaiting requested review from eccabay

@jeremyliweishih jeremyliweishih Awaiting requested review from jeremyliweishih

@ParthivNaresh ParthivNaresh Awaiting requested review from ParthivNaresh

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Make _get_prediction_vs_actual_data public and add to API reference
4 participants
@christopherbunn @dsherry @angela97lin @freddyaboulton