Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add new ensembler component #2653

Merged
merged 34 commits into from
Sep 23, 2021
Merged

Add new ensembler component #2653

merged 34 commits into from
Sep 23, 2021

Conversation

christopherbunn
Copy link
Contributor

Resolves #1930

@codecov
Copy link

codecov bot commented Aug 18, 2021 •
edited
Loading

Codecov Report

Merging #2653 (5359ef0) into main (5e62757) will increase coverage by 0.1%.
The diff coverage is 99.4%.

Impacted file tree graph

@@           Coverage Diff           @@
##            main   #2653     +/-   ##
=======================================
+ Coverage   99.8%   99.8%   +0.1%     
=======================================
  Files        297     302      +5     
  Lines      27741   28128    +387     
=======================================
+ Hits       27664   28050    +386     
- Misses        77      78      +1
Impacted Files Coverage Δ
evalml/automl/automl_search.py 99.9% <ø> (ø)
evalml/pipelines/__init__.py 100.0% <ø> (ø)
evalml/utils/gen_utils.py 99.1% <ø> (ø)
evalml/tests/component_tests/test_utils.py 95.3% <40.0%> (ø)
evalml/tests/automl_tests/test_automl_algorithm.py 97.4% <50.0%> (-2.6%) ⬇️
evalml/automl/automl_algorithm/automl_algorithm.py 100.0% <100.0%> (ø)
...valml/automl/automl_algorithm/default_algorithm.py 100.0% <100.0%> (ø)
...lml/automl/automl_algorithm/iterative_algorithm.py 100.0% <100.0%> (ø)
evalml/automl/engine/engine_base.py 100.0% <100.0%> (ø)
evalml/pipelines/component_graph.py 99.8% <100.0%> (ø)
... and 20 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5e62757...5359ef0. Read the comment docs.

@christopherbunn christopherbunn force-pushed the 1930_custom_ensembler branch 6 times, most recently from 9bc885c to 23fe8f9 Compare August 24, 2021 15:28
@christopherbunn christopherbunn marked this pull request as ready for review August 24, 2021 15:50
christopherbunn
christopherbunn commented Aug 24, 2021
View reviewed changes
evalml/tests/component_tests/test_utils.py Outdated
Comment on lines 42 to 48
n_components = 54
elif is_using_windows and not is_running_py_39_or_above:
# No prophet
n_components = 53
n_components = 55
elif is_using_windows and is_running_py_39_or_above:
# No detrender, no arima, no prophet
n_components = 51
n_components = 53
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since these are Windows/Conda tests, we might just not get coverage for this section until we merge to main.

@christopherbunn christopherbunn requested review from a team and angela97lin August 24, 2021 17:06
@christopherbunn christopherbunn force-pushed the 1930_custom_ensembler branch 7 times, most recently from 98daa6e to fad73b5 Compare September 3, 2021 16:41
@christopherbunn
Copy link
Contributor Author

christopherbunn commented Sep 7, 2021 •
edited
Loading

Here is the ensembler performance test doc that I have so far. I think that once I get in this PR for looking glass we can get into a deeper analysis of the results and see where the new ensembler ends up relative to the old one.

9/17 - Updated looking glass doc here:

eccabay
eccabay approved these changes Sep 7, 2021
View reviewed changes
Copy link
Contributor

@eccabay eccabay left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, this continues to be awesome work! Just left a couple small suggestions and nits.

evalml/automl/automl_algorithm/automl_algorithm.py Outdated Show resolved Hide resolved
evalml/automl/automl_algorithm/automl_algorithm.py Outdated Show resolved Hide resolved
evalml/automl/engine/engine_base.py Outdated Show resolved Hide resolved
evalml/pipelines/components/ensemble/stacked_ensemble_base.py Outdated Show resolved Hide resolved
evalml/pipelines/utils.py Outdated Show resolved Hide resolved
evalml/tests/component_tests/test_utils.py Outdated
# The total number of minimal components is 42
# The total number of components is 53
# The total number of minimal components is 43
# The total number of components is 56
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure 56 is the correct value?

@christopherbunn christopherbunn force-pushed the 1930_custom_ensembler branch 3 times, most recently from a7a13b7 to 5068338 Compare September 8, 2021 20:49
@christopherbunn
Removed extra elasticnet def
915a818
@christopherbunn
Fixed pipeline resolve and centralized ensemble builder
2a12a40
@christopherbunn
Updated partial dependence test to use new classifier
c0ae179
@christopherbunn
Added partial docstring and lint fixes
52b4428
@christopherbunn
Removed ensemble graph check and added test for different y
b2dac26
@christopherbunn
Added additional test and moved repeatable code into make ensemble pi…
ffea91a 
…peline function
@christopherbunn
Fixed partial dep test and lint fixes
667105a
@christopherbunn
Updated partial dep test
5a27783
@christopherbunn
Readded final estimator and updated tests again
0692d34
@christopherbunn
Fixed final estimator placement
2427127
@christopherbunn
Added in suggestions and lint fixes
7ef297d
@christopherbunn
Updated final estimator tests
a58a577
@christopherbunn
Updated naming schema and fixed test
95d8ec3
@christopherbunn
Change transform param to pass
7c59ce7
@christopherbunn
Explicitly copy component parameters to new ensemble pipeline
9a77d07
@christopherbunn
Rm extra import
6d56405
@christopherbunn
FIxed typo
4a7e92f
@christopherbunn
Code cleaning / updates
fcda1ae
@christopherbunn
Split off tests into two separate areas
56898fb
@christopherbunn
Updated nondefault y test case
13dece9
@christopherbunn
Fixed sklearn test case
77cc45e
@christopherbunn
Lint fixes
f889ca6
@christopherbunn
Lint fixes
e400b6b
@christopherbunn
Updated tests
fbe4100
@christopherbunn
Added ensembler input test
7e180b1
@christopherbunn
Removed extra import
5359ef0
@christopherbunn christopherbunn dismissed freddyaboulton’s stale review September 23, 2021 19:27

Addressed comments in this review.

@christopherbunn christopherbunn merged commit b01444a into main Sep 23, 2021
@freddyaboulton freddyaboulton deleted the 1930_custom_ensembler branch May 13, 2022 15:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@angela97lin angela97lin angela97lin approved these changes

@eccabay eccabay eccabay approved these changes

@chukarsten chukarsten chukarsten approved these changes

@freddyaboulton freddyaboulton Awaiting requested review from freddyaboulton

Assignees

@christopherbunn christopherbunn

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Build ensembler component
5 participants
@christopherbunn @angela97lin @freddyaboulton @eccabay @chukarsten