-
Notifications
You must be signed in to change notification settings - Fork 89
Freeze Hyperparameters for AutoMLSearch #1284
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Codecov Report
@@ Coverage Diff @@
## main #1284 +/- ##
==========================================
+ Coverage 99.95% 99.95% +0.01%
==========================================
Files 213 213
Lines 13857 13928 +71
==========================================
+ Hits 13850 13921 +71
Misses 7 7
Continue to review full report at Codecov.
|
try: | ||
self._tuners[pipeline.name].add(pipeline.parameters, score_to_minimize) | ||
except ValueError as e: | ||
if 'is not within the bounds of the space' in str(e): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Only throw the hyperparameter range error if we receive that error, otherwise we can throw the original error
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@bchen1116 looks great! I left a couple asks, and some suggestions on the docs content, but LGTM :D
…to bc_767_hyperparameters
@bchen1116 RE the bug you ran into with the docs here: I filed as #1367 We already know one workaround: create a woodwork |
fix #767
Right now, the implementation assumes that the user includes the 'default' values of the components in the hyperparameter dict. We catch the error that AutoML throws and warn the users if this default value isn't in the provided hyperparameter ranges. Hyperparameter validation would be tracked through this issue.
Since we have no hyperparameter validation as of yet, if a user creates a new pipeline with EvalML components, redefines a hyperparameter range that doesn't include the default value, and calls
AutoMLSearch
on this pipeline,AutoMLSearch
will raise the error that the default values aren't in the search space. This will ideally be addressed in the issue above. Additionally, since we don't do hyperparam validation, we don't catch/throw an error if a user misspells a component name, which will allowAutoMLSearch
to run with no errorsThis can be fixed once hyperparameter validation is implemented in the before-mentioned issue.
Docs here