Skip to content

Fix partial dependence graph method failing on multiclass problems when the label is numeric#2372

Merged
angela97lin merged 8 commits into
mainfrom
2114_partial_dep
Jun 15, 2021
Merged

Fix partial dependence graph method failing on multiclass problems when the label is numeric#2372
angela97lin merged 8 commits into
mainfrom
2114_partial_dep

Conversation

@angela97lin

Copy link
Copy Markdown
Contributor

Closes #2114

@angela97lin angela97lin self-assigned this Jun 10, 2021
@codecov

codecov Bot commented Jun 10, 2021

Copy link
Copy Markdown

Codecov Report

Merging #2372 (e37c756) into main (a096a7c) will increase coverage by 10.4%.
The diff coverage is 100.0%.

Impacted file tree graph

@@           Coverage Diff            @@
##            main   #2372      +/-   ##
========================================
+ Coverage   89.3%   99.7%   +10.4%     
========================================
  Files        281     281              
  Lines      24907   24926      +19     
========================================
+ Hits       22222   24829    +2607     
+ Misses      2685      97    -2588     
Impacted Files Coverage Δ
evalml/model_understanding/graphs.py 100.0% <100.0%> (+44.4%) ⬆️
...del_understanding_tests/test_partial_dependence.py 100.0% <100.0%> (+42.8%) ⬆️
evalml/automl/automl_search.py 99.4% <0.0%> (+0.2%) ⬆️
evalml/tests/utils_tests/test_logger.py 100.0% <0.0%> (+0.6%) ⬆️
evalml/pipelines/utils.py 99.1% <0.0%> (+1.0%) ⬆️
evalml/tests/conftest.py 99.6% <0.0%> (+1.1%) ⬆️
...derstanding/prediction_explanations/_algorithms.py 98.9% <0.0%> (+1.2%) ⬆️
...s/prediction_explanations_tests/test_explainers.py 100.0% <0.0%> (+1.8%) ⬆️
evalml/tests/component_tests/test_estimators.py 100.0% <0.0%> (+1.9%) ⬆️
evalml/tests/component_tests/test_components.py 100.0% <0.0%> (+2.8%) ⬆️
... and 42 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a096a7c...e37c756. Read the comment docs.

@angela97lin angela97lin marked this pull request as ready for review June 14, 2021 15:27
assert len(two_way_part_dep.columns) == grid_resolution + 1


def test_partial_dependence_multiclass_numeric_labels(

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@freddyaboulton I'm sure you might have some opinions about me adding yet another long running test 😂 I wasn't confident in a way to mock things but definitely would love suggestions/opinions!

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the runtime now? My takeaway is that using grid_resolution=5 can trim the test time to a couple seconds down from like ~10. The next step would be using fewer rows.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

~30-40 seconds before, 8 seconds after setting grid_resolution to 5 😁

@chukarsten chukarsten left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. I think dropping the grid res is a good idea to make it a shorter test.

@freddyaboulton freddyaboulton left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me @angela97lin !

@angela97lin angela97lin merged commit 1f7ccf9 into main Jun 15, 2021
@angela97lin angela97lin deleted the 2114_partial_dep branch June 15, 2021 00:34
@chukarsten chukarsten mentioned this pull request Jun 22, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Partial Dependence Graph fails on multiclass problems when the label is numeric

3 participants