ReplaceNullableTypes Component #3090

chukarsten · 2021-11-22T22:16:27Z

Addresses #3057

codecov · 2021-11-22T22:20:52Z

Codecov Report

Merging #3090 (168fa5c) into main (6a82b55) will increase coverage by 0.1%.
The diff coverage is 100.0%.

@@           Coverage Diff           @@
##            main   #3090     +/-   ##
=======================================
+ Coverage   99.7%   99.7%   +0.1%     
=======================================
  Files        315     317      +2     
  Lines      30547   30679    +132     
=======================================
+ Hits       30447   30579    +132     
  Misses       100     100

Impacted Files	Coverage Δ
evalml/pipelines/components/__init__.py	`100.0% <ø> (ø)`
...alml/pipelines/components/transformers/__init__.py	`100.0% <ø> (ø)`
evalml/tests/component_tests/test_utils.py	`95.7% <ø> (ø)`
.../components/transformers/preprocessing/__init__.py	`100.0% <100.0%> (ø)`
...ansformers/preprocessing/replace_nullable_types.py	`100.0% <100.0%> (ø)`
evalml/tests/component_tests/test_components.py	`99.0% <100.0%> (ø)`
...ts/component_tests/test_nullable_types_replacer.py	`100.0% <100.0%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6a82b55...168fa5c. Read the comment docs.

evalml/pipelines/components/transformers/preprocessing/replace_nullable_types.py

chukarsten · 2021-12-02T16:02:28Z

Makefile

@@ -85,6 +85,14 @@ installdeps:
 	pip install --upgrade pip -q
 	pip install -e . -q

+.PHONY: installdeps-min


A convenience to help people debug the minimum dependency checks we have in CI.

We should use this in the CI job? Maybe in a follow-up pr.

This is already done in the github workflows, specifically the min deps CI workflows. I just put it in here so people can build a test environment to run them. If there's a better way that people do it, I can take this out!

evalml/pipelines/components/transformers/preprocessing/replace_nullable_types.py

chukarsten · 2021-12-02T16:25:27Z

evalml/tests/component_tests/test_components.py

@@ -831,7 +832,8 @@ def test_component_has_random_seed():
        assert "random_seed" in params


-def test_transformer_transform_output_type(X_y_binary):
+@pytest.mark.parametrize("component_class", _all_transformers())


Just modified this test to move the for-loop to pytest parameters. Made it easier on test failure to understand which transformer component was failing.

chukarsten · 2021-12-02T16:25:56Z

evalml/tests/component_tests/test_components.py

-                    type(y),
-                    y.name if isinstance(y, pd.Series) else None,
-                )
+    if component_class in [PolynomialDetrender, LogTransformer, LabelEncoder]:


This should all be de-indenting...

chukarsten · 2021-12-02T16:27:15Z

evalml/tests/component_tests/test_nullable_types_replacer.py

+
+@pytest.mark.parametrize("nullable_types_properly_set", [True, False])
+@pytest.mark.parametrize("input_type", ["ww", "pandas"])
+def test_replace_nullable_types_integer_target(


Initially, the boolean and integer target tests were together, but it resulted in too much branch-complexity and ultimately made the tests harder to understand.

freddyaboulton

@chukarsten Thank you for this! I think the behavior and testing is solid. I left some minor comments on how to stream-line the implementation.

evalml/pipelines/components/transformers/preprocessing/replace_nullable_types.py

evalml/tests/component_tests/test_nullable_types_replacer.py

freddyaboulton · 2021-12-02T21:03:23Z

Makefile

@@ -85,6 +85,14 @@ installdeps:
 	pip install --upgrade pip -q
 	pip install -e . -q

+.PHONY: installdeps-min


We should use this in the CI job? Maybe in a follow-up pr.

evalml/pipelines/components/transformers/preprocessing/replace_nullable_types.py

…inal behavior of running through the infer_feature_types.

…ra conversion.

…lacenullable transformer and update tests.

… is set and properly.

chukarsten force-pushed the nullable_types_preprocessing branch from 75522be to 980167b Compare November 22, 2021 22:20

chukarsten marked this pull request as ready for review November 23, 2021 15:02

auto-assign bot assigned chukarsten Nov 23, 2021

chukarsten requested review from freddyaboulton, angela97lin, bchen1116, christopherbunn, eccabay, jeremyliweishih, jlouns and ParthivNaresh November 23, 2021 15:03

chukarsten marked this pull request as draft November 23, 2021 15:11

freddyaboulton reviewed Nov 23, 2021

View reviewed changes

evalml/pipelines/components/transformers/preprocessing/replace_nullable_types.py Outdated Show resolved Hide resolved

freddyaboulton reviewed Nov 23, 2021

View reviewed changes

evalml/pipelines/components/transformers/preprocessing/replace_nullable_types.py Outdated Show resolved Hide resolved

chukarsten force-pushed the nullable_types_preprocessing branch from 583bde2 to 43aa896 Compare November 29, 2021 18:31

ParthivNaresh reviewed Nov 30, 2021

View reviewed changes

chukarsten force-pushed the nullable_types_preprocessing branch from 02d8af0 to cf845fc Compare December 2, 2021 05:30

chukarsten commented Dec 2, 2021

View reviewed changes

chukarsten marked this pull request as ready for review December 2, 2021 16:38

freddyaboulton suggested changes Dec 2, 2021

View reviewed changes

gsheni requested review from jeff-hernandez and thehomebrewnerd December 2, 2021 21:57

tamargrey reviewed Dec 3, 2021

View reviewed changes

evalml/pipelines/components/transformers/preprocessing/replace_nullable_types.py Outdated Show resolved Hide resolved

evalml/pipelines/components/transformers/preprocessing/replace_nullable_types.py Show resolved Hide resolved

thehomebrewnerd reviewed Dec 3, 2021

View reviewed changes

chukarsten force-pushed the nullable_types_preprocessing branch from 36565bc to 19f7178 Compare December 3, 2021 16:01

chukarsten requested review from freddyaboulton, tamargrey, thehomebrewnerd and ParthivNaresh December 4, 2021 02:44

chukarsten added 25 commits December 7, 2021 13:07

Added some woodwork stuff.

d771359

Reverted the behavior of the transform() and had it preserve the orig…

11feebf

…inal behavior of running through the infer_feature_types.

Modified docstring.

d795360

Got target impl written and tested.

388ba77

Experimental.

3b2c369

Lint.

9064692

Pydocstyle.

d1983a6

Lint.

a9a42a5

Added a min dependency installer.

6b4e906

Fixes for 3.7 minimum deps.

749c862

Fixed more 3.7 min dep tests.

03ccb38

Lint.

a8b0446

Removed accidental ignore_nullable_types.

5a8feff

Addressed some PR review changes.

f1d61cb

Linted up and removed some of the unnecessary functions.

b2417ac

Added the tests for AgeNullable and the woodwork typing tests.

7906935

Docstring cleanup and removed unnecessary init_series.

8b2cc0e

Addressed comment.

280531d

Made the logical_type arg more consistent and removed unnecessary ext…

c5ceb6a

…ra conversion.

Reverted the base transform fit_transform, implemented it for the rep…

511903f

…lacenullable transformer and update tests.

Lint.

3f38c54

Added an exercise of the target woodwork accessor to make sure schema…

3567f83

… is set and properly.

Got rid of unused imports.

38db0cb

Lint.

1458f85

Updated release.

8ea2cce

chukarsten force-pushed the nullable_types_preprocessing branch from 93054a5 to 8ea2cce Compare December 7, 2021 18:07

Merge branch 'main' into nullable_types_preprocessing

168fa5c

chukarsten merged commit 16e7e67 into main Dec 7, 2021

chukarsten mentioned this pull request Dec 9, 2021

Release v0.39.0 #3136

Merged

freddyaboulton deleted the nullable_types_preprocessing branch May 13, 2022 15:23

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ReplaceNullableTypes Component #3090

ReplaceNullableTypes Component #3090

chukarsten commented Nov 22, 2021

codecov bot commented Nov 22, 2021 •

edited

Loading

chukarsten Dec 2, 2021

freddyaboulton Dec 2, 2021

chukarsten Dec 3, 2021

chukarsten Dec 2, 2021

chukarsten Dec 2, 2021

chukarsten Dec 2, 2021

freddyaboulton left a comment

freddyaboulton Dec 2, 2021

ReplaceNullableTypes Component #3090

ReplaceNullableTypes Component #3090

Conversation

chukarsten commented Nov 22, 2021

codecov bot commented Nov 22, 2021 • edited Loading

Codecov Report

chukarsten Dec 2, 2021

Choose a reason for hiding this comment

freddyaboulton Dec 2, 2021

Choose a reason for hiding this comment

chukarsten Dec 3, 2021

Choose a reason for hiding this comment

chukarsten Dec 2, 2021

Choose a reason for hiding this comment

chukarsten Dec 2, 2021

Choose a reason for hiding this comment

chukarsten Dec 2, 2021

Choose a reason for hiding this comment

freddyaboulton left a comment

Choose a reason for hiding this comment

freddyaboulton Dec 2, 2021

Choose a reason for hiding this comment

codecov bot commented Nov 22, 2021 •

edited

Loading