Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Limit model usage for EN, XBG, CB for large multiclass problems #2982

Merged
merged 20 commits into from Nov 2, 2021

Conversation

bchen1116
Copy link
Contributor

@bchen1116 bchen1116 commented Oct 27, 2021

fix #2976

@bchen1116 bchen1116 self-assigned this Oct 27, 2021
@codecov
Copy link

codecov bot commented Oct 27, 2021

Codecov Report

Merging #2982 (caa7c2c) into main (981024f) will increase coverage by 0.1%.
The diff coverage is 100.0%.

Impacted file tree graph

@@           Coverage Diff           @@
##            main   #2982     +/-   ##
=======================================
+ Coverage   99.7%   99.7%   +0.1%     
=======================================
  Files        309     309             
  Lines      29312   29399     +87     
=======================================
+ Hits       29221   29308     +87     
  Misses        91      91             
Impacted Files Coverage Δ
...lml/automl/automl_algorithm/iterative_algorithm.py 100.0% <100.0%> (ø)
evalml/automl/automl_search.py 99.9% <100.0%> (+0.1%) ⬆️
.../automl_tests/test_automl_search_classification.py 100.0% <100.0%> (ø)
...lml/tests/automl_tests/test_iterative_algorithm.py 100.0% <100.0%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 981024f...caa7c2c. Read the comment docs.

Copy link
Contributor

@chukarsten chukarsten left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for knocking this one out!! A few things I'd like to see before we merge this: a little more documentation as noted in the review and a logger statement to appear in the verbose output of AutoML search. I think with that, in addition to all the good stuff you currently have, and this behavior is well documented and communicated clearly. Good call also making the filter_estimators function. We may need to conduct a similar test with the DefaultAlgorithm and move that function out of the IterativeAlgorithm to be used communally. An issue is filed for that : #2985 .

evalml/automl/automl_algorithm/iterative_algorithm.py Outdated Show resolved Hide resolved
docs/source/release_notes.rst Outdated Show resolved Hide resolved
Copy link
Contributor

@freddyaboulton freddyaboulton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @bchen1116 ! Left some minor comments but otherwise looks great to me. I agree with the other threads on this pr as well.

Copy link
Contributor

@chukarsten chukarsten left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks gucci! Thanks for addressing. Sorry for the long wait on re-review.

@bchen1116 bchen1116 merged commit b7b64b9 into main Nov 2, 2021
@chukarsten chukarsten mentioned this pull request Nov 9, 2021
@freddyaboulton freddyaboulton deleted the bc_2976_limit_estimators branch May 13, 2022 15:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Limit EN, XGB, CB estimators on large searches, allow user opt-in
4 participants