Upgrade WW to 0.4.1 #2379

jeremyliweishih · 2021-06-14T19:55:01Z

This PR mostly consists of fixing unit tests and functionality where we deal with WW logical types since they are now instantiated instead of returning the class.

Perf tests look good with no changes in model performance and somelittle change in init time and fit time but all within a couple percentage points. Ran with 50 iterations and 3 trials.

ww_upgrade_perf_results.zip

codecov · 2021-06-15T14:26:34Z

Codecov Report

Merging #2379 (ea5eb8e) into main (461076b) will increase coverage by 0.1%.
The diff coverage is 100.0%.

@@           Coverage Diff           @@
##            main   #2379     +/-   ##
=======================================
+ Coverage   99.7%   99.7%   +0.1%     
=======================================
  Files        283     283             
  Lines      25240   25244      +4     
=======================================
+ Hits       25140   25144      +4     
  Misses       100     100

Impacted Files	Coverage Δ
...components/transformers/imputers/simple_imputer.py	`100.0% <ø> (ø)`
...ta_checks_tests/test_invalid_targets_data_check.py	`100.0% <ø> (ø)`
evalml/data_checks/invalid_targets_data_check.py	`100.0% <100.0%> (ø)`
evalml/data_checks/target_leakage_data_check.py	`100.0% <100.0%> (ø)`
evalml/model_understanding/graphs.py	`100.0% <100.0%> (ø)`
...rmers/preprocessing/delayed_feature_transformer.py	`100.0% <100.0%> (ø)`
evalml/pipelines/utils.py	`99.1% <100.0%> (ø)`
evalml/tests/automl_tests/test_automl.py	`99.8% <100.0%> (ø)`
.../tests/component_tests/test_datetime_featurizer.py	`100.0% <100.0%> (ø)`
...mponent_tests/test_delayed_features_transformer.py	`100.0% <100.0%> (ø)`
... and 16 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 461076b...ea5eb8e. Read the comment docs.

core-requirements.txt

jeremyliweishih · 2021-06-21T14:00:45Z

evalml/tests/component_tests/test_datetime_featurizer.py

            X = X_df.copy()
            X.ww.init(logical_types={0: logical_type})
-        except (ww.exceptions.TypeConversionError, TypeError):
+        except (ww.exceptions.TypeConversionError, TypeError, ValueError):


will be fixed by alteryx/woodwork#991

Thanks for filing!

tamargrey · 2021-06-21T15:17:28Z

evalml/data_checks/invalid_targets_data_check.py


        y = infer_feature_types(y)
-        is_supported_type = y.ww.logical_type in numeric_and_boolean_ww + [
+        is_supported_type = type(y.ww.logical_type) in numeric_and_boolean_ww + [


I think this can be isinstance(y.ww.logical_type, tuple(numeric_and_boolean_ww)) if you'd like here and in the other places where a similar check.

@tamargrey I think the other option is to use y.ww.logical_type.type_string and then update our lists, equality/member checks to use the type_strings. What are your thoughts? I think both do the same thing but using type_string may be backwards compatible?

@freddyaboulton I like that idea! It does seem like it would be backwards compatible, which is nice.

It's definitely different from how we've tended to compare logical types inside of Woodwork, so my one worry is that it's using the type_string for something that it wasn't intended for. I'm not sure that that's actually a problem, though.

tamargrey · 2021-06-21T15:22:08Z

evalml/model_understanding/graphs.py

+        is_type = type(X.ww.logical_types[X.columns[feature]]) == ltype
    else:
-        is_type = X.ww.logical_types[feature] == ltype
+        is_type = type(X.ww.logical_types[feature]) == ltype


These are also places where we can use isinstance, which I think is slightly better from a timing perspective than == checks.

freddyaboulton

@jeremyliweishih Thanks for this! I think it looks great. I have a question on the diff for _retain_woodwork_types... and I think it may be better to follow @tamargrey 's suggestion to use is_instance or my suggestion to use type_string rather than wrapping everything in type. I'm ok with either of those options but @tamargrey 's opinion should take precedence over mine hehe.

freddyaboulton · 2021-06-21T16:30:33Z

core-requirements.txt

 shap>=0.36.0
 texttable>=1.6.2
-woodwork==0.3.1
+woodwork>=0.4.1


We can close PR #2373 right?

freddyaboulton · 2021-06-21T16:41:47Z

evalml/tests/component_tests/test_datetime_featurizer.py

            X = X_df.copy()
            X.ww.init(logical_types={0: logical_type})
-        except (ww.exceptions.TypeConversionError, TypeError):
+        except (ww.exceptions.TypeConversionError, TypeError, ValueError):


Thanks for filing!

freddyaboulton · 2021-06-21T17:42:02Z

evalml/utils/woodwork_utils.py

    if ltypes_to_ignore is None:
        ltypes_to_ignore = []
+    old_logical_types = {
+        k: v if inspect.isclass(v) else type(v) for k, v in old_logical_types.items()


Do need isclass here? The types should always be instances from now on right?

IIRC some tests were failing without this change but I'll double check this!

freddyaboulton · 2021-06-21T17:48:36Z

evalml/data_checks/invalid_targets_data_check.py


        y = infer_feature_types(y)
-        is_supported_type = y.ww.logical_type in numeric_and_boolean_ww + [
+        is_supported_type = type(y.ww.logical_type) in numeric_and_boolean_ww + [


@tamargrey I think the other option is to use y.ww.logical_type.type_string and then update our lists, equality/member checks to use the type_strings. What are your thoughts? I think both do the same thing but using type_string may be backwards compatible?

freddyaboulton · 2021-06-21T17:52:57Z

FYI I'll file a follow-up issue to use the new functionality added in this release:

Add option to return TableSchema instead of DataFrame from table accessor select method (#916)
Add dropping and renaming columns inplace (#920)

jeremyliweishih · 2021-06-21T17:58:56Z

@freddyaboulton can you also add using concat_columns to that issue?

dsherry · 2021-06-22T19:23:10Z

Taking a slight calculated risk and merging this before the docs build completes. We have a lot queued right now. I hope #2404 will help us here!

The build_conda_pkg failure is expected, due to the version mismatch with the latest branch in the conda feedstock repo.

jeremyliweishih added 7 commits June 14, 2021 10:07

Change to 4.0.0

6fe93be

Add first few select and fix logical types

1950c2d

RL

801873c

Merge branch 'main' of github.com:alteryx/evalml into js_2327_ww

c4ed731

lint

1a1f996

remove return schema for now

009d1f4

Merge branch 'main' of github.com:alteryx/evalml into js_2327_ww

9ddb75e

jeremyliweishih added 2 commits June 15, 2021 14:00

Add to min requirements as well

5173be2

Fix datetime feat tests

b9ed7ff

gsheni reviewed Jun 16, 2021

View reviewed changes

core-requirements.txt Outdated Show resolved Hide resolved

jeremyliweishih added 19 commits June 16, 2021 16:40

Fix delayed feature transformer test

62b1f61

Fix drop null

deca960

Fix featuretools tests

3ca0441

Fix feature selector test

c6e60b2

Fix component tests

19dc59f

More component tests

89f835c

Fix LSA tests

bf564b5

Fix OHE tests

0cbc8e2

Fix imputer tests

c1fb7b1

Fix retain type ww

1a91753

fix target encoder tests'

ea50541

lint

45c5d46

Merge branch 'main' of github.com:alteryx/evalml into js_2327_ww

cc58e85

Fix invalid target data check

cc995ea

Fix component graph tests

54603c7

Merge branch 'main' of github.com:alteryx/evalml into js_2327_ww

ea84138

lint

4a297d4

Revert formatting of these files

e5922e1

Fix target leakage check

9c5ab9d

jeremyliweishih requested review from ParthivNaresh, angela97lin, bchen1116, christopherbunn and gsheni June 21, 2021 13:59

gsheni requested review from tamargrey and removed request for gsheni June 21, 2021 13:59

jeremyliweishih commented Jun 21, 2021

View reviewed changes

tamargrey reviewed Jun 21, 2021

View reviewed changes

Merge branch 'main' into js_2327_ww

610c1fd

freddyaboulton approved these changes Jun 21, 2021

View reviewed changes

jeremyliweishih added 12 commits June 21, 2021 14:15

Test without util change;

b35c90d

Merge branch 'main' of github.com:alteryx/evalml into js_2327_ww

3b21294

Merge branch 'js_2327_ww' of github.com:alteryx/evalml into js_2327_ww

cda71a1

lint

a5f10c6

Switch to is instance

c174fad

Use type_string in numeric_and_boolean_ww

ae4bb84

Use isinstance when applicable

1d39e28

Merge branch 'main' of github.com:alteryx/evalml into js_2327_ww

5134f23

Merge branch 'main' into js_2327_ww

a0ba58b

Merge branch 'main' into js_2327_ww

f9beffe

Merge branch 'main' into js_2327_ww

1c310f3

Merge branch 'main' into js_2327_ww

ea5eb8e

dsherry merged commit 7487189 into main Jun 22, 2021

chukarsten mentioned this pull request Jun 22, 2021

Release v0.27.0 #2428

Merged

freddyaboulton mentioned this pull request Jun 29, 2021

Use some of woodwork's latest features #2457

Closed

5 tasks

freddyaboulton deleted the js_2327_ww branch May 13, 2022 15:34

Upgrade WW to 0.4.1 #2379

Upgrade WW to 0.4.1 #2379

Uh oh!

Conversation

jeremyliweishih commented Jun 14, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Jun 15, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

freddyaboulton left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

freddyaboulton commented Jun 21, 2021

Uh oh!

jeremyliweishih commented Jun 21, 2021

Uh oh!

dsherry commented Jun 22, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

jeremyliweishih commented Jun 14, 2021 •

edited

Loading

codecov bot commented Jun 15, 2021 •

edited

Loading

dsherry commented Jun 22, 2021 •

edited

Loading