Fully separate numeric and categorical data in `DefaultAlgorithm` by jeremyliweishih · Pull Request #3209 · alteryx/evalml

jeremyliweishih · 2022-01-10T22:33:25Z

Example:

You can see that a one hot encoder does not exist in the numeric pipeline.

Perf test results:
split_fixed_no_change.html.zip

…plit

jeremyliweishih · 2022-01-10T22:35:59Z

    def __init__(self, column_types=None, random_seed=0, **kwargs):
        parameters = {"column_types": column_types}
        parameters.update(kwargs)
-        Transformer.__init__(


This was causing parameters to disappear. Still slightly unsure of the exact mechanism of how the parameters disappear but my hypothesis is that the parameters field in SelectByType you can see here is different from the parameters field created by this init logic.

It's not that they're disappearing, it's that they're put in the parameters dictionary because that's that the Transformer init does.

'Numeric Pipeline - Select Columns By Type Transformer': {'columns': None, 'parameters': {'column_types': ['numeric']}, 'component_obj': None}

I agree with the redesign. ColumnSelectors are meant to take a list of columns so we need to either need to rename column_types to columns or not make this component a subclass. What we here doing before of trying to skip the parent init and get to the grandparent init is probably not the right thing to do.

Interesting find!

jeremyliweishih · 2022-01-10T22:36:25Z



-class SelectByType(ColumnSelector):
+class SelectByType(Transformer):


Decided to just inherit from Transformer and override fit as well.

always good to consider

jeremyliweishih · 2022-01-10T22:38:16Z

             Defaults to None
-         extra_components (list[ComponentBase]): List of extra components to be added after preprocessing components. Defaults to None.
-         extra_components_position (str): Where to put extra components. Defaults to "before_preprocessing" and any other value will put components after preprocessing components.
+         extra_components_before (list[ComponentBase]): List of extra components to be added before preprocessing components. Defaults to None.


Revised this API here to just have two lists for components in both locations. I was thinking about changing it to a component to index mapping but thought it would be too complicated (I would need to figure out index locations and how they would change as we insert each component).

I think this is a solid change. I think you should let the use cases drive the API. Do we currently have (or foresee) a reason to be able to inject components at different locations?

I don't see any short term usage other than in DefaultAlgorithm but I can imagine sometime in the future we would want more control over where exact components go in our pipelines. I agree that we should let the use cases drive the API so let's leave this for now!

codecov · 2022-01-10T22:38:38Z

Codecov Report

Merging #3209 (6c90a07) into main (1a3f32e) will increase coverage by 0.1%.
The diff coverage is 100.0%.

@@           Coverage Diff           @@
##            main   #3209     +/-   ##
=======================================
+ Coverage   99.8%   99.8%   +0.1%     
=======================================
  Files        326     326             
  Lines      31497   31563     +66     
=======================================
+ Hits       31406   31472     +66     
  Misses        91      91

Impacted Files	Coverage Δ
evalml/automl/automl_algorithm/automl_algorithm.py	`100.0% <100.0%> (ø)`
...valml/automl/automl_algorithm/default_algorithm.py	`100.0% <100.0%> (ø)`
...elines/components/transformers/column_selectors.py	`92.9% <100.0%> (ø)`
evalml/pipelines/utils.py	`99.4% <100.0%> (-<0.1%)`	⬇️
...valml/tests/automl_tests/test_default_algorithm.py	`100.0% <100.0%> (ø)`
...mponent_tests/test_column_selector_transformers.py	`100.0% <100.0%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1a3f32e...6c90a07. Read the comment docs.

jeremyliweishih · 2022-01-10T22:38:59Z

        }
        numeric_pipeline = make_pipeline(
-            self.X,
+            self.X.ww.select(exclude="category"),


here I exclude all category semantic tag columns so there won't be preprocessing components for those components.

jeremyliweishih · 2022-01-10T22:39:12Z

            }
            categorical_pipeline = make_pipeline(
-                self.X,
+                self.X.ww.select(include=["category"]),


Likewise I do the opposite here.

…plit

chukarsten

Looks good, Jeremy! Just a copy pasta and a nit. Do with them what you want!

chukarsten · 2022-01-12T16:53:08Z



-class SelectByType(ColumnSelector):
+class SelectByType(Transformer):


always good to consider

chukarsten · 2022-01-12T17:29:38Z

             Defaults to None
-         extra_components (list[ComponentBase]): List of extra components to be added after preprocessing components. Defaults to None.
-         extra_components_position (str): Where to put extra components. Defaults to "before_preprocessing" and any other value will put components after preprocessing components.
+         extra_components_before (list[ComponentBase]): List of extra components to be added before preprocessing components. Defaults to None.


I think this is a solid change. I think you should let the use cases drive the API. Do we currently have (or foresee) a reason to be able to inject components at different locations?

…plit

…c pipeline

…plit

…js_3020_fully_split

eccabay

This is really cool work! Just left a few small nits

ParthivNaresh

Excellent work, the perf tests look great!

…plit

jeremyliweishih added 6 commits January 10, 2022 13:49

Add DropByType

36acc07

Remove Drop By Type

387e418

Add SelectByType component into numeric pipeline

567c2c5

Fix Select By Type init so parameters pass properly

c086090

Merge branch 'main' of github.com:alteryx/evalml into js_3020_fully_s…

4b78961

…plit

RL

25322cd

jeremyliweishih commented Jan 10, 2022

View reviewed changes

revert util changes

1464579

jeremyliweishih commented Jan 10, 2022

View reviewed changes

jeremyliweishih added 2 commits January 10, 2022 17:40

fix wrong component in tests'

5994078

Merge branch 'main' of github.com:alteryx/evalml into js_3020_fully_s…

70b5723

…plit

jeremyliweishih marked this pull request as ready for review January 11, 2022 21:20

auto-assign Bot assigned jeremyliweishih Jan 11, 2022

chukarsten approved these changes Jan 12, 2022

View reviewed changes

jeremyliweishih added 4 commits January 12, 2022 13:21

Cache cat/non-cat data

d3be713

Merge branch 'main' of github.com:alteryx/evalml into js_3020_fully_s…

136a5b1

…plit

Fix tests

2aed9be

Docstring changes

d34f43b

freddyaboulton reviewed Jan 13, 2022

View reviewed changes

Comment thread evalml/automl/automl_algorithm/default_algorithm.py Outdated

Fix typo

3b34b75

freddyaboulton reviewed Jan 13, 2022

View reviewed changes

Comment thread evalml/automl/automl_algorithm/default_algorithm.py Outdated

freddyaboulton reviewed Jan 13, 2022

View reviewed changes

Comment thread evalml/automl/automl_algorithm/default_algorithm.py

jeremyliweishih added 4 commits January 13, 2022 15:06

Remove check for column type input

a042e1f

Add exclude parameter for SelectByType and exclude category in numeri…

e86a4bd

…c pipeline

Merge branch 'main' of github.com:alteryx/evalml into js_3020_fully_s…

626d629

…plit

Fix RL

5a753b8

jeremyliweishih and others added 9 commits January 18, 2022 10:25

Merge branch 'main' into js_3020_fully_split

b0d719c

Refactor to remove assumption of having numeric pipeline

0bf290a

Merge branch 'main' of github.com:alteryx/evalml into js_3020_fully_s…

0ffe61c

…plit

Update make pipeline calls with correct X

de74124

Merge branch 'main' into js_3020_fully_split

21bbc68

Use full X for non-split case

7f0d1f3

Add split pipeline only categorical case

ded0568

Add categorical and numeric only cases to test_default_algorithm

787492b

Merge branch 'js_3020_fully_split' of github.com:alteryx/evalml into …

bdcba58

…js_3020_fully_split

jeremyliweishih requested review from ParthivNaresh, angela97lin, bchen1116, christopherbunn, chukarsten, dsherry, eccabay and freddyaboulton January 18, 2022 20:43

jeremyliweishih added 2 commits January 21, 2022 11:01

Merge branch 'main' into js_3020_fully_split

2e93c29

Merge branch 'main' into js_3020_fully_split

e5828aa

eccabay reviewed Jan 24, 2022

View reviewed changes

Comment thread docs/source/release_notes.rst Outdated

Comment thread evalml/pipelines/components/transformers/column_selectors.py Outdated

ParthivNaresh approved these changes Jan 24, 2022

View reviewed changes

Comment thread docs/source/release_notes.rst Outdated

Comment thread evalml/automl/automl_algorithm/default_algorithm.py Outdated

Comment thread evalml/tests/automl_tests/test_default_algorithm.py Outdated

Comment thread evalml/tests/automl_tests/test_default_algorithm.py

jeremyliweishih added 6 commits January 24, 2022 11:36

Merge branch 'main' into js_3020_fully_split

3c9c3ba

Update release notes

6b9f5fa

Remove uncessary check for input call

d7f6f01

remove uncessary if statement

9bf8498

cleanup

b19827a

Merge branch 'main' of github.com:alteryx/evalml into js_3020_fully_s…

6c90a07

…plit

jeremyliweishih merged commit 2cb356d into main Jan 24, 2022

angela97lin mentioned this pull request Jan 25, 2022

Release 0.43.0 #3279

Merged

freddyaboulton deleted the js_3020_fully_split branch May 13, 2022 15:34



		class SelectByType(ColumnSelector):
		class SelectByType(Transformer):

Conversation

jeremyliweishih commented Jan 10, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov Bot commented Jan 10, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

chukarsten left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

eccabay left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

ParthivNaresh left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

jeremyliweishih commented Jan 10, 2022 •

edited

Loading

codecov Bot commented Jan 10, 2022 •

edited

Loading