Skip to content

Limit model usage for EN, XBG, CB for large multiclass problems - #2982

Merged
bchen1116 merged 20 commits into
mainfrom
bc_2976_limit_estimators
Nov 2, 2021
Merged

bchen1116 merged 20 commits into
mainfrom
bc_2976_limit_estimators

Conversation

@bchen1116

@bchen1116 bchen1116 commented Oct 27, 2021 •

Copy link
Copy Markdown
Contributor

fix #2976

@bchen1116 bchen1116 self-assigned this Oct 27, 2021
@codecov

codecov Bot commented Oct 27, 2021 •

Copy link
Copy Markdown

Codecov Report

Merging #2982 (caa7c2c) into main (981024f) will increase coverage by 0.1%.
The diff coverage is 100.0%.

Impacted file tree graph

@@           Coverage Diff           @@
##            main   #2982     +/-   ##
=======================================
+ Coverage   99.7%   99.7%   +0.1%     
=======================================
  Files        309     309             
  Lines      29312   29399     +87     
=======================================
+ Hits       29221   29308     +87     
  Misses        91      91             
Impacted Files Coverage Δ
...lml/automl/automl_algorithm/iterative_algorithm.py 100.0% <100.0%> (ø)
evalml/automl/automl_search.py 99.9% <100.0%> (+0.1%) ⬆️
.../automl_tests/test_automl_search_classification.py 100.0% <100.0%> (ø)
...lml/tests/automl_tests/test_iterative_algorithm.py 100.0% <100.0%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 981024f...caa7c2c. Read the comment docs.

Comment thread evalml/automl/automl_algorithm/iterative_algorithm.py

@chukarsten chukarsten left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for knocking this one out!! A few things I'd like to see before we merge this: a little more documentation as noted in the review and a logger statement to appear in the verbose output of AutoML search. I think with that, in addition to all the good stuff you currently have, and this behavior is well documented and communicated clearly. Good call also making the filter_estimators function. We may need to conduct a similar test with the DefaultAlgorithm and move that function out of the IterativeAlgorithm to be used communally. An issue is filed for that : #2985 .

Comment thread evalml/automl/automl_algorithm/iterative_algorithm.py Outdated
Comment thread docs/source/release_notes.rst Outdated
Comment thread evalml/automl/automl_algorithm/iterative_algorithm.py
Comment thread evalml/automl/automl_algorithm/iterative_algorithm.py

@freddyaboulton freddyaboulton left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @bchen1116 ! Left some minor comments but otherwise looks great to me. I agree with the other threads on this pr as well.

Comment thread evalml/tests/automl_tests/test_automl_search_classification.py Outdated
Comment thread evalml/tests/automl_tests/test_iterative_algorithm.py Outdated
Comment thread evalml/tests/automl_tests/test_automl_search_classification.py
Comment thread evalml/tests/automl_tests/test_iterative_algorithm.py

@chukarsten chukarsten left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks gucci! Thanks for addressing. Sorry for the long wait on re-review.

@bchen1116
bchen1116 merged commit b7b64b9 into main Nov 2, 2021
@chukarsten chukarsten mentioned this pull request Nov 9, 2021
@freddyaboulton
freddyaboulton deleted the bc_2976_limit_estimators branch May 13, 2022 15:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Limit EN, XGB, CB estimators on large searches, allow user opt-in

4 participants