Update automl to use default of max_batches=1 #1452
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
dsherry
force-pushed
the
ds_update_max_batches_default
branch
from
2379738
to
276defe
Compare
Codecov Report
@@ Coverage Diff @@
## main #1452 +/- ##
=========================================
+ Coverage 100.0% 100.0% +0.1%
=========================================
Files 223 223
Lines 15001 15013 +12
=========================================
+ Hits 14994 15006 +12
Misses 7 7
Continue to review full report at Codecov.
|
dsherry
added
documentation
dsherry
force-pushed
the
ds_update_max_batches_default
branch
from
9b41693
to
aac9988
Compare
dsherry
requested review from
freddyaboulton,
angela97lin,
christopherbunn,
bchen1116 and
ParthivNaresh
docs/source/user_guide/automl.ipynb
Outdated
| @@ -74,7 +74,7 @@ | |||
| "source": [ | |||
| "The AutoML search will log its progress, reporting each pipeline and parameter set evaluated during the search.\n", | |||
| "\n", | |||
| "There are a number of mechanisms to control the AutoML search time. One way is to set the maximum number of candidate models to be evaluated during AutoML using `max_iterations`. By default, AutoML will search a fixed number of iterations and parameter pairs (`max_iterations=5`). The first pipeline to be evaluated will always be a baseline model representing a trivial solution. " | |||
| "There are a number of mechanisms to control the AutoML search time. One way is to set the `max_batches` parameter which controls the maximum number of rounds of AutoML to evaluate, where each round may train and score a variable number of pipeline. Another way is to set the `max_iterations` parameter which controls the maximum number of candidate models to be evaluated during AutoML. By default, AutoML will search for a single batch. The first pipeline to be evaluated will always be a baseline model representing a trivial solution. " | |||
There was a problem hiding this comment.
Not related to this PR but when are we making _pipelines_per_batch public?
There was a problem hiding this comment.
@freddyaboulton yeah good q, no plans to do so currently, let's chat at standup
dsherry
force-pushed
the
ds_update_max_batches_default
branch
from
f74ab35
to
afae52e
Compare
bchen1116
approved these changes
Nov 24, 2020
angela97lin
reviewed
Nov 24, 2020
| if not isinstance(max_time, (int, float, str, type(None))): | ||
| raise TypeError(f"Parameter max_time must be a float, int, string or None. Received {str(max_time)}.") | ||
| if isinstance(max_time, (int, float)) and max_time <= 0: | ||
| raise ValueError(f"Parameter max_time must be None or non-negative. Received {max_time}.") |
There was a problem hiding this comment.
Hmm, the ValueError message doesn't align with the check (max_time <= 0 vs non-negative)
(Same with max_batches and max_iterations)
There was a problem hiding this comment.
Maybe 'strictly positive' rather than 'non-negative'?
There was a problem hiding this comment.
Good point. I'll change the boundary conditions here to be non-negative and if we wanna change this in the future we can!
angela97lin
reviewed
Nov 24, 2020
dsherry
force-pushed
the
ds_update_max_batches_default
branch
from
afae52e
to
06f826e
Compare
freddyaboulton
approved these changes
Nov 24, 2020
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We have 8 classification estimators and 7 regression estimators now! We should use all of them by default.
Docs changes visible here.