Move ROC and confusion matrix to pipeline plot utils (#696) #704
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## master #704 +/- ##
==========================================
+ Coverage 99.09% 99.19% +0.10%
==========================================
Files 139 140 +1
Lines 4954 4952 -2
==========================================
+ Hits 4909 4912 +3
+ Misses 45 40 -5
Continue to review full report at Codecov.
|
|
I rekicked the RTD job for this branch again: I also rekicked All passed with flying colors. I also followed the instructions on #586 to use the RTD docker container to build the docs locally. That succeeded without any issues. I was able to view the fresh html files and they looked fine. |
* Impl * Rename file * Update test * Forgot to add the new file * Fix imports and lint * Fix test * Changelog * Fix api docs * Add missing markdown descr. Delete some ipynb elements which were failing validation * Update docstrings * Delete MSLE for codecov * Increase test coverage for codecov * Disallow unsupported options for normalize_confusion_matrix * Add test coverage for get_objective w\ nonetype (for codecov) * Update docstring * Remove unnecessary raise in test mock * Update test. * Update docstring
angela97lin
approved these changes
Apr 24, 2020
dsherry
force-pushed
the
ds_639_clean_up_roc_confusion_take2
branch
from
9a64d36 to
87f2f2e
Compare
|
I hope so too! I just rebased. I'll watch RTD pass once more, and then merge. |
|
The RTD build passed and the docs look good. Merging. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Preface: this is a clone of PR #696 -- I merged that to master this morning, but ran into RTD timeouts (described in #702). Reset master and now trying again.
Fix #639 #365 #392 #427 #608 #610 #620
Prior to #346 , plot data functions like ROC and confusion matrix were treated as objectives. #346 and follow-on work separated them from objectives, and made it so that these plot data functions aren't precomputed during automl. We then filed epic #639 to figure out where to put the code for plot data functions like ROC and confusion matrix.
This PR has the following changes:
evalml/pipelines/plot_utils.pyas standalone functions which take actual vs predictedplot_metricsinput from automl, and deleteplot_datafield from automl results, because those were made unnecessary when we took ROC/confusion out of automlgen_utilsmethods to the API docs (likeimport_or_raise)MSLEobjective for codecov -- it was disabled with a commentFuture work:
ClassificationPipeline.confusion_matrix,ClassificationPipeline.plot_confusion_matrix,BinaryClassificationPipeline.roc_curve,BinaryClassificationPipeline.plot_roc_curve. This would make it convenient to get plot data directly from pipelines