Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update AutoSearchBase to use dynamically generated preprocessing pipelines #870

Merged
merged 35 commits into from Jun 26, 2020

Conversation

angela97lin
Copy link
Contributor

@angela97lin angela97lin commented Jun 22, 2020

Closes #844

Notes:
self.allowed_pipelines isn't instantiated until search() if not supplied as a parameter, is that weird? (same with self.allowed_model_families)

@angela97lin angela97lin marked this pull request as draft Jun 22, 2020
@codecov
Copy link

codecov bot commented Jun 23, 2020

Codecov Report

Merging #870 into master will increase coverage by 0.00%.
The diff coverage is 100.00%.

Impacted file tree graph

@@           Coverage Diff            @@
##           master     #870    +/-   ##
========================================
  Coverage   99.76%   99.77%            
========================================
  Files         193      193            
  Lines        8603     8830   +227     
========================================
+ Hits         8583     8810   +227     
  Misses         20       20            
Impacted Files Coverage Δ
...lml/automl/automl_algorithm/iterative_algorithm.py 100.00% <100.00%> (ø)
evalml/automl/automl_search.py 98.36% <100.00%> (+0.04%) ⬆️
evalml/pipelines/utils.py 100.00% <100.00%> (ø)
evalml/tests/automl_tests/test_automl.py 100.00% <100.00%> (ø)
.../automl_tests/test_automl_search_classification.py 100.00% <100.00%> (ø)
...ests/automl_tests/test_automl_search_regression.py 100.00% <100.00%> (ø)
evalml/tests/conftest.py 100.00% <100.00%> (ø)
evalml/tests/pipeline_tests/test_pipelines.py 99.81% <100.00%> (+<0.01%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1e2fcb5...eab1100. Read the comment docs.

@angela97lin angela97lin requested review from kmax12 and dsherry Jun 24, 2020
@angela97lin angela97lin marked this pull request as ready for review Jun 24, 2020
evalml/pipelines/utils.py Outdated Show resolved Hide resolved
evalml/pipelines/utils.py Outdated Show resolved Hide resolved
kmax12
kmax12 approved these changes Jun 25, 2020
Copy link
Contributor

@kmax12 kmax12 left a comment

i tested on a local dataset and it work and did a quick pass through the code and left some comments. has my approval, but id get one more.

evalml/automl/auto_search_base.py Outdated Show resolved Hide resolved
evalml/automl/auto_search_base.py Outdated Show resolved Hide resolved
evalml/automl/auto_search_base.py Outdated Show resolved Hide resolved
evalml/pipelines/utils.py Outdated Show resolved Hide resolved
Copy link
Collaborator

@dsherry dsherry left a comment

@angela97lin I didn't review the tests yet, will do. But I left comments on the impl. Looks good! Mostly minor stuff. Ping me whenever you feel this is ready for re-review and I'll read the whole thing.

docs/source/changelog.rst Show resolved Hide resolved
evalml/automl/auto_search_base.py Outdated Show resolved Hide resolved
evalml/automl/auto_search_base.py Outdated Show resolved Hide resolved
evalml/automl/auto_search_base.py Outdated Show resolved Hide resolved
evalml/automl/auto_search_base.py Outdated Show resolved Hide resolved
evalml/automl/auto_search_base.py Outdated Show resolved Hide resolved
evalml/pipelines/utils.py Outdated Show resolved Hide resolved
evalml/pipelines/utils.py Show resolved Hide resolved
evalml/pipelines/utils.py Show resolved Hide resolved
Copy link
Collaborator

@dsherry dsherry left a comment

Looks good! Left suggestions on splitting up the tests, parametrize, cleaning up dead code in the impl and logging. I'd say the only blocking thing is deleting the dead code, but would be great to get the other stuff in too, either in this PR or separate

evalml/automl/auto_search_base.py Outdated Show resolved Hide resolved
evalml/automl/auto_search_base.py Outdated Show resolved Hide resolved
evalml/automl/auto_search_base.py Outdated Show resolved Hide resolved
evalml/pipelines/utils.py Show resolved Hide resolved
@angela97lin
Copy link
Contributor Author

angela97lin commented Jun 26, 2020

@dsherry FYI I split the tests up as you suggested, and parameterized some but not all, my reason being that it got really unwieldy to add in all of the different parameters (dataset, mocks, mock pipelines) to support binary/multiclass and this seemed cleaner. We can always revisit though!

@angela97lin angela97lin merged commit bf9867b into master Jun 26, 2020
2 checks passed
@angela97lin angela97lin deleted the 844_autobase branch Jun 26, 2020
@angela97lin angela97lin mentioned this pull request Jun 30, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Update AutoSearchBase to use dynamically generated preprocessing pipelines
3 participants