Upgrade to WW 0.16.2 by freddyaboulton · Pull Request #3482 · alteryx/evalml

freddyaboulton · 2022-04-26T15:56:34Z

Pull Request Description

Fix all the test failures caused by the latest woodwork version.

After creating the pull request: in order to pass the release_notes_updated check you will need to update the "Future Release" section of docs/source/release_notes.rst to include this pull request by adding :pr:123.

codecov · 2022-04-26T16:03:20Z

Codecov Report

Merging #3482 (8838328) into main (881c1e0) will decrease coverage by 0.8%.
The diff coverage is 100.0%.

@@           Coverage Diff           @@
##            main   #3482     +/-   ##
=======================================
- Coverage   99.7%   98.9%   -0.7%     
=======================================
  Files        336     336             
  Lines      33368   33364      -4     
=======================================
- Hits       33242   32985    -257     
- Misses       126     379    +253

Impacted Files	Coverage Δ
evalml/data_checks/multicollinearity_data_check.py	`100.0% <ø> (ø)`
evalml/data_checks/target_leakage_data_check.py	`100.0% <ø> (ø)`
evalml/tests/data_checks_tests/test_data_checks.py	`100.0% <100.0%> (ø)`
..._checks_tests/test_multicollinearity_data_check.py	`100.0% <100.0%> (ø)`
...ata_checks_tests/test_target_leakage_data_check.py	`100.0% <100.0%> (ø)`
evalml/automl/pipeline_search_plots.py	`17.9% <0.0%> (-82.1%)`	⬇️
...l/tests/automl_tests/test_pipeline_search_plots.py	`23.3% <0.0%> (-76.7%)`	⬇️
...ests/automl_tests/test_automl_search_regression.py	`74.7% <0.0%> (-20.7%)`	⬇️
.../automl_tests/test_automl_search_classification.py	`84.5% <0.0%> (-12.1%)`	⬇️
evalml/tests/automl_tests/test_automl_utils.py	`91.1% <0.0%> (-8.9%)`	⬇️
... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 881c1e0...8838328. Read the comment docs.

freddyaboulton · 2022-04-26T19:37:09Z

        - requirements-parser >=0.2.0
        - shap >=0.40.0
        - texttable >=1.6.2
-        - woodwork >=0.15.0


0.16.2 not available for download yet which is why the build conda pkg is failing. Should be fixed later though

freddyaboulton · 2022-04-26T19:37:49Z


-    collinear_cols = [c for c in X.columns if "not" not in c]
+    collinear_cols = [
+        c for c in X.columns if c not in {"not_collinear", "col_nullable"}


Nullable types are now handled differently

freddyaboulton · 2022-04-26T19:38:27Z

-    y = y.astype(bool)
+    X["d"] = y % 2
+    X["e"] = [0] * 30
+    X.ww.init(logical_types={"d": "Boolean"})


This seems to be related to this bug: alteryx/woodwork#1404

good catch - was having a lot of trouble with this one..

freddyaboulton · 2022-04-26T19:39:40Z

    if input_type == "ww":
        y = ww.init_series(y)
        y_no_variance = ww.init_series(y_no_variance)
-    null_leakage = [


The problem here is that the actual mutual information calculation changed this release so the scores are now 0.6 as opposed to >0.95 and there's no way to change that from the default data checks.

should we put up an issue to track adding this coverage back? I'm pretty meh on this since we already have coverage on the unit tests for each check. It's also very painful to find or manipulate the existing test data to fail all these checks.

I think we're ok since we already have coverage for all the checks!

freddyaboulton · 2022-04-26T19:40:42Z

-    X["d"] = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
-    X["e"] = ["a", "b", "c", "d", "e", "f", "g", "h", "i", "j", "k", "l", "m", "n", "o"]
-    X.ww.init(logical_types={"e": "categorical"})
+    X["d"] = np.zeros(len(y))


Deleted column e because of this ticket: alteryx/woodwork#1405

jeremyliweishih

LGTM - great catches on the problematic test failures 👏

jeremyliweishih · 2022-04-26T19:43:15Z

            Columns in X that are highly correlated with each other will be identified using mutual information.

-            >>> col = pd.Series([1, 0, 2, 3, 4])
+            >>> col = pd.Series([1, 0, 2, 3, 4] * 15)


this is way smarter than what I was doing 🥲

jeremyliweishih · 2022-04-26T19:45:16Z

    if input_type == "ww":
        y = ww.init_series(y)
        y_no_variance = ww.init_series(y_no_variance)
-    null_leakage = [


should we put up an issue to track adding this coverage back? I'm pretty meh on this since we already have coverage on the unit tests for each check. It's also very painful to find or manipulate the existing test data to fail all these checks.

jeremyliweishih · 2022-04-26T19:46:41Z

-    y = y.astype(bool)
+    X["d"] = y % 2
+    X["e"] = [0] * 30
+    X.ww.init(logical_types={"d": "Boolean"})


good catch - was having a lot of trouble with this one..

jeremyliweishih · 2022-04-26T19:48:09Z


-    y = pd.Series([1, 0, 1, 1])
+    y = pd.Series(
+        [1, 0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1, 1, 0, 1, 1, 1]


nit: you could replace this with the [1, 0, 1] * X format if you wanted!

jeremyliweishih and others added 3 commits April 25, 2022 11:39

try to fix tests

f7c6d05

Fix tests

b4c0938

Update requirements files

dea361d

freddyaboulton changed the title ~~Upgrade to WW 0.16.1~~ Upgrade to WW 0.16.2 Apr 26, 2022

freddyaboulton added 4 commits April 26, 2022 12:22

Update doctests

f3a55c1

Add to release notes

4a4e979

Update latest dependency versions

21eeed5

Fix pandas version

da0c769

freddyaboulton commented Apr 26, 2022

View reviewed changes

freddyaboulton marked this pull request as ready for review April 26, 2022 19:40

auto-assign Bot assigned freddyaboulton Apr 26, 2022

freddyaboulton requested review from ParthivNaresh, bchen1116, christopherbunn, chukarsten, eccabay, fjlanasa and jeremyliweishih and removed request for ParthivNaresh and eccabay April 26, 2022 19:41

jeremyliweishih approved these changes Apr 26, 2022

View reviewed changes

Fix test cast construction

8838328

freddyaboulton enabled auto-merge (squash) April 27, 2022 14:19

freddyaboulton merged commit b0e0946 into main Apr 27, 2022

freddyaboulton deleted the js_upgrade_ww_0.16.1 branch April 27, 2022 15:17

chukarsten mentioned this pull request Apr 29, 2022

Release v0.51.0. #3489

Merged

Conversation

freddyaboulton commented Apr 26, 2022

Pull Request Description

Uh oh!

codecov Bot commented Apr 26, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jeremyliweishih left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codecov Bot commented Apr 26, 2022 •

edited

Loading