Updating scikit-learn and scikit-optimize to latest version #1141
Conversation
Codecov Report
@@ Coverage Diff @@
## main #1141 +/- ##
=======================================
Coverage 99.92% 99.92%
=======================================
Files 196 196
Lines 11999 12006 +7
=======================================
+ Hits 11990 11997 +7
Misses 9 9
Continue to review full report at Codecov.
|
freddyaboulton
requested review from
angela97lin,
jeremyliweishih,
eccabay and
bchen1116
|
Performance tests forthcoming |
freddyaboulton
force-pushed
the
update-sklearn-skopt-dependencies
branch
from
d796b4d
to
2160637
Compare
|
Performance Test Results here. Scores stayed the same but the fit times have increased slightly. I think it's still ok to merge this. |
freddyaboulton
force-pushed
the
update-sklearn-skopt-dependencies
branch
from
2160637
to
a96b34a
Compare
freddyaboulton
force-pushed
the
update-sklearn-skopt-dependencies
branch
6 times, most recently
from
70dac87
to
384a765
Compare
| scikit-learn>=0.21.3,!=0.22,<0.23.0 | ||
| scikit-optimize>=0.7,<=0.7.4 | ||
| scikit-learn>=0.23 | ||
| scikit-optimize>=0.8 |
There was a problem hiding this comment.
What a relief! 😁 This was getting ugly.
| @@ -199,10 +200,11 @@ class MockRegressor(Estimator): | |||
| name = "Mock Regressor" | |||
| model_family = ModelFamily.NONE | |||
| supported_problem_types = [ProblemTypes.REGRESSION] | |||
| hyperparameter_ranges = {} | |||
| hyperparameter_ranges = {'a': Integer(0, 10), | |||
| 'b': Real(0, 10)} | |||
There was a problem hiding this comment.
@freddyaboulton I have the same question as @bchen1116 . You mentioned
the latest version of scikit-optimize wont optimize a pipeline without parameters
So which test(s) does setting these ranges affect? I don't think I follow yet.
There was a problem hiding this comment.
In test_automl_search_classification.py and test_automl_search_regression.py we pass in dummy_regression_pipeline_class, dummy_binary_pipeline_class, or dummy_multiclass_pipeline_class as the allowed_pipelines parameter to AutoMLSearch. Since these pipelines don't have parameters (because the estimator doesn't have any parameters) the SKOpt tuner can't propose any pipelines and the search fails. Since we mock the pipeline fit and score methods, adding these parameters doesn't change the behavior of the tests.
The other option is to mock SKOptTuner.propose in all tests that would fail but that seemed like a bigger change.
freddyaboulton
force-pushed
the
update-sklearn-skopt-dependencies
branch
2 times, most recently
from
e0eba46
to
db9ad97
Compare
…o increase coverage.
freddyaboulton
force-pushed
the
update-sklearn-skopt-dependencies
branch
from
db9ad97
to
478a23b
Compare
Pull Request Description
Fixes #1121 . With the release of
scikit-optimize0.8.0, we can update toscikit-learnversion >= 0.23 (Release Notes).The only catch is that now
SKOptTunercan't propose a new pipeline if the pipeline has no parameters. I had to add dummy parameters to our estimator fixtures to address this.We may want to add a check to
AutoMLthat verifies the pipelines specified inallowed_pipelineshave parameters. This can be done in #454 or in a separate issue.After creating the pull request: in order to pass the release_notes_updated check you will need to update the "Future Release" section of
docs/source/release_notes.rstto include this pull request by adding :pr:123.