Freeze Hyperparameters for AutoMLSearch#1284
Conversation
Codecov Report
@@ Coverage Diff @@
## main #1284 +/- ##
==========================================
+ Coverage 99.95% 99.95% +0.01%
==========================================
Files 213 213
Lines 13857 13928 +71
==========================================
+ Hits 13850 13921 +71
Misses 7 7
Continue to review full report at Codecov.
|
| try: | ||
| self._tuners[pipeline.name].add(pipeline.parameters, score_to_minimize) | ||
| except ValueError as e: | ||
| if 'is not within the bounds of the space' in str(e): |
There was a problem hiding this comment.
Only throw the hyperparameter range error if we receive that error, otherwise we can throw the original error
There was a problem hiding this comment.
@bchen1116 looks great! I left a couple asks, and some suggestions on the docs content, but LGTM :D
…to bc_767_hyperparameters
|
@bchen1116 RE the bug you ran into with the docs here: I filed as #1367 We already know one workaround: create a woodwork |
fix #767
Right now, the implementation assumes that the user includes the 'default' values of the components in the hyperparameter dict. We catch the error that AutoML throws and warn the users if this default value isn't in the provided hyperparameter ranges. Hyperparameter validation would be tracked through this issue.
Since we have no hyperparameter validation as of yet, if a user creates a new pipeline with EvalML components, redefines a hyperparameter range that doesn't include the default value, and calls
AutoMLSearchon this pipeline,AutoMLSearchwill raise the error that the default values aren't in the search space. This will ideally be addressed in the issue above. Additionally, since we don't do hyperparam validation, we don't catch/throw an error if a user misspells a component name, which will allowAutoMLSearchto run with no errorsThis can be fixed once hyperparameter validation is implemented in the before-mentioned issue.
Docs here