Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Only show best results for a given pipeline template in rankings #682

Merged
merged 13 commits into from Apr 21, 2020

Conversation

angela97lin
Copy link
Contributor

@angela97lin angela97lin commented Apr 20, 2020

Closes #410

@codecov
Copy link

codecov bot commented Apr 20, 2020

Codecov Report

Merging #682 into master will increase coverage by 0.00%.
The diff coverage is 100.00%.

Impacted file tree graph

@@           Coverage Diff           @@
##           master     #682   +/-   ##
=======================================
  Coverage   99.08%   99.09%           
=======================================
  Files         139      139           
  Lines        4934     4947   +13     
=======================================
+ Hits         4889     4902   +13     
  Misses         45       45           
Impacted Files Coverage Δ
evalml/automl/auto_base.py 96.40% <100.00%> (+0.03%) ⬆️
...ts/automl_tests/test_auto_classification_search.py 100.00% <100.00%> (ø)
evalml/tests/automl_tests/test_autobase.py 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 297c077...fb5b83a. Read the comment docs.

@angela97lin angela97lin requested a review from dsherry Apr 21, 2020
@angela97lin angela97lin marked this pull request as ready for review Apr 21, 2020
docs/source/changelog.rst Outdated Show resolved Hide resolved
evalml/automl/auto_base.py Outdated Show resolved Hide resolved
evalml/automl/auto_base.py Outdated Show resolved Hide resolved
Copy link
Collaborator

@dsherry dsherry left a comment

Great start! Left some comments and questions

assert isinstance(automl.best_pipeline, PipelineBase)
assert isinstance(automl.get_pipeline(0), PipelineBase)

automl.describe_pipeline(0)

scores = automl.best_pipeline.score(X, y, ['precision'])
assert not any(np.isnan(val) for val in scores.values())
Copy link
Collaborator

@dsherry dsherry Apr 21, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool. I think you can also do assert not np.isnan(np.array(scores.values())).any()

Copy link
Contributor Author

@angela97lin angela97lin Apr 21, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tried and I think it would be assert not np.isnan(np.array(list(scores.values()))).any(), not sure if there's a clear better statement here :p

Copy link
Collaborator

@dsherry dsherry Apr 21, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lol no worries, was just a passing comment. I've found numpy methods to generally be faster than the python builtins.

@angela97lin angela97lin requested a review from dsherry Apr 21, 2020
Copy link
Collaborator

@dsherry dsherry left a comment

Left one more suggestion, otherwise 🚢 !

@angela97lin angela97lin merged commit d1da30b into master Apr 21, 2020
2 checks passed
@angela97lin angela97lin deleted the 410_best_pipeline_results branch Apr 21, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Only show best results for a given pipeline template in rankings
3 participants