Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes UnboundLocalError: local variable 'cv_pipeline' referenced before assignment when error in automl search #996

Merged
merged 10 commits into from Jul 31, 2020

Conversation

angela97lin
Copy link
Contributor

@angela97lin angela97lin commented Jul 30, 2020

Closes #995

Before this PR, in the unlikely chance that _compute_cv_scores errors before setting cv_pipeline = pipeline.clone(), then we get an UnboundLocalError: local variable 'cv_pipeline' referenced before assignment in the later check if isinstance(cv_pipeline, BinaryClassificationPipeline) and cv_pipeline.threshold is not None. This PR addresses this by initializing it to None first before the try/except block.
`

@angela97lin angela97lin self-assigned this Jul 30, 2020
@codecov
Copy link

codecov bot commented Jul 30, 2020

Codecov Report

Merging #996 into main will increase coverage by 0.19%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #996      +/-   ##
==========================================
+ Coverage   99.67%   99.86%   +0.19%     
==========================================
  Files         179      179              
  Lines        9424     9436      +12     
==========================================
+ Hits         9393     9423      +30     
+ Misses         31       13      -18     
Impacted Files Coverage Δ
...alml/pipelines/prediction_explanations/__init__.py 100.00% <ø> (ø)
evalml/automl/automl_search.py 99.55% <100.00%> (+0.45%) ⬆️
evalml/tests/automl_tests/test_automl.py 100.00% <100.00%> (ø)
evalml/tests/component_tests/test_components.py 99.57% <0.00%> (+0.42%) ⬆️
.../automl_tests/test_automl_search_classification.py 100.00% <0.00%> (+0.45%) ⬆️
evalml/tests/pipeline_tests/test_pipelines.py 100.00% <0.00%> (+0.82%) ⬆️
...ests/automl_tests/test_automl_search_regression.py 100.00% <0.00%> (+1.06%) ⬆️
evalml/utils/gen_utils.py 100.00% <0.00%> (+2.53%) ⬆️
... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 93e7ebe...f36609e. Read the comment docs.

@angela97lin angela97lin marked this pull request as ready for review July 30, 2020 17:10
@angela97lin angela97lin requested review from dsherry, freddyaboulton, eccabay and jeremyliweishih and removed request for dsherry July 30, 2020 17:10
Copy link
Contributor

@freddyaboulton freddyaboulton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@angela97lin Looks good to me! I left a comment about how to write a unit test for this. Apart from that, this made it clear to me that we don't have a good way of catching errors that come from splitting the data. That might be intended but if not we should discuss what to do about that.

evalml/automl/automl_search.py Show resolved Hide resolved
@angela97lin
Copy link
Contributor Author

@freddyaboulton Agreed, we don't have a clear way of telling the user that the error stemmed from splitting data, but with this in place, at least we'll default to our catching / logging of pipeline errors :D

Copy link
Contributor

@dsherry dsherry left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@angela97lin great reproducer on the issue, and your explanation of the problem on here was helpful! This rocks. I left one comment about checking for nan scores in the test.

Copy link
Contributor

@jeremyliweishih jeremyliweishih left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM -agree with Dylan's comment on checking for nan

@angela97lin angela97lin merged commit 2bd3563 into main Jul 31, 2020
@angela97lin angela97lin mentioned this pull request Jul 31, 2020
@angela97lin angela97lin added this to the July 2020 milestone Jul 31, 2020
@angela97lin angela97lin deleted the 995_err branch September 24, 2020 15:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

UnboundLocalError: local variable 'cv_pipeline' referenced before assignment when error in automl search
4 participants