Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix ensembling for generate_pipeline_example #4102

Merged
merged 33 commits into from
Mar 23, 2023

Conversation

jeremyliweishih
Copy link
Collaborator

Fixes #4101

@codecov
Copy link

codecov bot commented Mar 22, 2023

Codecov Report

Merging #4102 (c709013) into main (eb621d9) will increase coverage by 15.2%.
The diff coverage is 100.0%.

@@           Coverage Diff            @@
##            main   #4102      +/-   ##
========================================
+ Coverage   84.6%   99.7%   +15.2%     
========================================
  Files        349     349              
  Lines      37644   37661      +17     
========================================
+ Hits       31812   37542    +5730     
+ Misses      5832     119    -5713     
Impacted Files Coverage Δ
evalml/pipelines/__init__.py 100.0% <ø> (ø)
evalml/pipelines/components/__init__.py 100.0% <ø> (ø)
evalml/pipelines/component_graph.py 99.8% <100.0%> (+0.1%) ⬆️
evalml/pipelines/pipeline_base.py 98.5% <100.0%> (-<0.1%) ⬇️
evalml/pipelines/utils.py 99.6% <100.0%> (+1.8%) ⬆️
evalml/tests/pipeline_tests/test_pipeline_utils.py 99.6% <100.0%> (+0.1%) ⬆️

... and 33 files with indirect coverage changes

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@jeremyliweishih jeremyliweishih marked this pull request as ready for review March 22, 2023 21:23

pipeline = aml.get_pipeline(2)
if ensembling:
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ensembling pipelines do not work as each select columns transformer is not fit and will return no columns when using the test env. This also fails for any catboost or xgboost pipeline. Will file an issue to patch this behavior in AutoMLTestEnv but wanted to get this out first.

@jeremyliweishih jeremyliweishih requested review from eccabay, chukarsten, christopherbunn and tamargrey and removed request for eccabay March 23, 2023 15:00
@@ -445,7 +450,10 @@ def _transform_features(
if not fit:
X_schema = (
self._return_non_engineered_features(X).ww.schema
if "DFS Transformer" in self.compute_order
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this mean we were getting the input types wrong for stacked ensemble pipeline with the dfs transformer present?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes - this was not working for any ensemble with DFS transformer 😢

Copy link
Contributor

@christopherbunn christopherbunn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Smaller nits but once the questions Tamar put up are addressed it LGTM

evalml/pipelines/utils.py Show resolved Hide resolved
evalml/tests/pipeline_tests/test_pipeline_utils.py Outdated Show resolved Hide resolved
evalml/pipelines/component_graph.py Outdated Show resolved Hide resolved
@jeremyliweishih jeremyliweishih enabled auto-merge (squash) March 23, 2023 20:03
@jeremyliweishih jeremyliweishih merged commit c7054d8 into main Mar 23, 2023
@jeremyliweishih jeremyliweishih deleted the js_4101_fix_ensembling_example branch March 23, 2023 20:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Fix ensembling for generate_pipeline_example
5 participants