Add `DateTimeNaNDataCheck` #2039

jeremyliweishih · 2021-03-25T20:15:29Z

Fixes #1999.

codecov · 2021-03-25T20:29:04Z

Codecov Report

Merging #2039 (79fff01) into main (8556450) will increase coverage by 0.1%.
The diff coverage is 100.0%.

@@            Coverage Diff            @@
##             main    #2039     +/-   ##
=========================================
+ Coverage   100.0%   100.0%   +0.1%     
=========================================
  Files         280      282      +2     
  Lines       22934    23004     +70     
=========================================
+ Hits        22925    22995     +70     
  Misses          9        9

Impacted Files	Coverage Δ
evalml/data_checks/__init__.py	`100.0% <100.0%> (ø)`
evalml/data_checks/data_check_message_code.py	`100.0% <100.0%> (ø)`
evalml/data_checks/datetime_nan_data_check.py	`100.0% <100.0%> (ø)`
evalml/data_checks/default_data_checks.py	`100.0% <100.0%> (ø)`
evalml/tests/data_checks_tests/test_data_checks.py	`100.0% <100.0%> (ø)`
.../data_checks_tests/test_datetime_nan_data_check.py	`100.0% <100.0%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8556450...79fff01. Read the comment docs.

jeremyliweishih · 2021-03-25T22:00:09Z

evalml/data_checks/default_data_checks.py

@@ -19,11 +20,12 @@ class DefaultDataChecks(DataChecks):
        - `InvalidTargetDataCheck`
        - `NoVarianceDataCheck`
        - `ClassImbalanceDataCheck` (for classification problem types)
+        - `DateTimeNaNDataCheck`

    """

    _DEFAULT_DATA_CHECK_CLASSES = [HighlyNullDataCheck, IDColumnsDataCheck,


Since AutoMLSearch will error out if a user provides datetime columns and has NaN values, I opted to add to default data checks as a safe guard.

Didn't realize that don't have data checks as part of AutoMLSearch now. Open to removing this as well.

I think it's fine to leave in for now, as we can still use the default data checks with DefaultDataChecks().validate(X_train, y_train). As our data check process updates, this might change, but I see no issues in adding it here

jeremyliweishih · 2021-03-25T22:00:55Z

evalml/data_checks/datetime_nan_data_check.py

+error_contains_nan = "Input datetime column ({}) contains NaN values. Please input NaN values or drop this column."
+
+
+class DateTimeNaNDataCheck(DataCheck):


Do we have a standardization of NaN vs Null terminology? I decided to use NaN here but can change to null as well.

That's a good question...I don't think nulls fall into the scope of this. We have data checks to alert us to highly null columns and I don't think a null by itself makes the algorithm choke.

bchen1116

Code looks good to me! I left a few suggestions on tests to add to solidify our coverage a bit more, and I also agree that adding this datacheck to DefaultDataChecks is fine. PR should be good to go after the added tests though! 👌

bchen1116 · 2021-03-26T13:31:44Z

evalml/data_checks/datetime_nan_data_check.py

+    """Checks if datetime columns contain NaN values."""
+
+    def __init__(self):
+        """Checks each column in the input for datetime features and will issue and error if NaN values are present.


typo: will issue an error

bchen1116 · 2021-03-26T13:38:53Z

evalml/data_checks/default_data_checks.py

@@ -19,11 +20,12 @@ class DefaultDataChecks(DataChecks):
        - `InvalidTargetDataCheck`
        - `NoVarianceDataCheck`
        - `ClassImbalanceDataCheck` (for classification problem types)
+        - `DateTimeNaNDataCheck`

    """

    _DEFAULT_DATA_CHECK_CLASSES = [HighlyNullDataCheck, IDColumnsDataCheck,


I think it's fine to leave in for now, as we can still use the default data checks with DefaultDataChecks().validate(X_train, y_train). As our data check process updates, this might change, but I see no issues in adding it here

evalml/tests/data_checks_tests/test_datetime_nan_data_check.py

bchen1116 · 2021-03-26T13:49:01Z

Ah, and also add this new data check to the docs

CLAassistant · 2021-03-26T14:35:07Z

All committers have signed the CLA.

ParthivNaresh

Looks great man

evalml/data_checks/default_data_checks.py

freddyaboulton

@jeremyliweishih Implementation looks solid! I agree with the suggestions to add coverage!

evalml/data_checks/datetime_nan_data_check.py

evalml/tests/data_checks_tests/test_datetime_nan_data_check.py

chukarsten

I think besides the test coverage suggested by @bchen1116 , there's a few copy pastas from the highly null data check you used for a template ;). Otherwise, good work!

chukarsten · 2021-03-26T19:37:12Z

evalml/data_checks/datetime_nan_data_check.py

+error_contains_nan = "Input datetime column ({}) contains NaN values. Please input NaN values or drop this column."
+
+
+class DateTimeNaNDataCheck(DataCheck):


That's a good question...I don't think nulls fall into the scope of this. We have data checks to alert us to highly null columns and I don't think a null by itself makes the algorithm choke.

evalml/tests/data_checks_tests/test_datetime_nan_data_check.py

angela97lin

A bit of copy pasta and testing updates needed, but I also think it might be useful to combine all the columns into one for the error. Unlike the other data checks that return granular info per col, this only reports which datetime cols have nan values, so seems cleaner to just combine.

evalml/data_checks/__init__.py

evalml/data_checks/datetime_nan_data_check.py

jeremyliweishih · 2021-03-29T23:03:07Z

evalml/tests/data_checks_tests/test_data_checks.py

-                                                 data_check_name="NoVarianceDataCheck",
-                                                 message_code=DataCheckMessageCode.NO_VARIANCE,
-                                                 details={"column": "Y"}).to_dict()],
+        "errors": messages[4:7] + [DataCheckError(message="Y has 1 unique value.",


@angela97lin

…999_dtf_nan

stale

jeremyliweishih added 4 commits March 25, 2021 15:12

Add datetime_nan_datacheck

a905cbb

Add datetime_nan_data_check tests

7e861b4

RL

090b689

Merge branch 'main' of github.com:alteryx/evalml into js_1999_dtf_nan

7048062

jeremyliweishih added 6 commits March 25, 2021 17:09

Add to API reference

4d23386

Add doc test

128c068

Merge branch 'main' of github.com:alteryx/evalml into js_1999_dtf_nan

9bc46d4

Add better msg

e58548a

Add to default data checks

2efb047

Lint

5c33ff7

jeremyliweishih marked this pull request as ready for review March 25, 2021 21:58

auto-assign bot assigned jeremyliweishih Mar 25, 2021

jeremyliweishih commented Mar 25, 2021

View reviewed changes

Merge branch 'main' into js_1999_dtf_nan

c03367f

jeremyliweishih requested review from angela97lin, freddyaboulton, bchen1116, chukarsten and ParthivNaresh March 25, 2021 23:02

bchen1116 approved these changes Mar 26, 2021

View reviewed changes

ParthivNaresh approved these changes Mar 26, 2021

View reviewed changes

evalml/data_checks/default_data_checks.py Show resolved Hide resolved

freddyaboulton approved these changes Mar 26, 2021

View reviewed changes

evalml/data_checks/datetime_nan_data_check.py Outdated Show resolved Hide resolved

evalml/tests/data_checks_tests/test_datetime_nan_data_check.py Show resolved Hide resolved

chukarsten previously requested changes Mar 26, 2021

View reviewed changes

angela97lin previously requested changes Mar 28, 2021

View reviewed changes

evalml/data_checks/__init__.py Outdated Show resolved Hide resolved

evalml/data_checks/datetime_nan_data_check.py Outdated Show resolved Hide resolved

jeremyliweishih added 2 commits March 29, 2021 11:03

Fix copy pasta

de1ea1f

Fix error msg

9602cc1

jeremyliweishih added 12 commits March 29, 2021 11:06

Fix spacing

ec014b3

one error

f852467

add more test cases

7d17f95

Merge branch 'main' of github.com:alteryx/evalml into js_1999_dtf_nan

d36453f

Add to default datacheck test

5b555f0

Fix doctest

1064d2b

Fix datacheck test

65021d6

Add nan to regression default datachecks

9369872

Add nan_dt_col leakage warning

82a0314

Fix default tests

5dca691

Fix tests

b17ae74

Merge branch 'main' of github.com:alteryx/evalml into js_1999_dtf_nan

1cd46d2

jeremyliweishih commented Mar 29, 2021

View reviewed changes

jeremyliweishih added 7 commits March 29, 2021 19:05

typo

c9c41ce

Add to_dict() to datetime_nan_data_check tests

8f29cbf

Fix doctest

fb7319c

Merge branch 'main' into js_1999_dtf_nan

bec461b

Add to docs

77ef3db

Merge branch 'js_1999_dtf_nan' of github.com:alteryx/evalml into js_1…

8aa57fe

…999_dtf_nan

typo

79fff01

jeremyliweishih requested review from chukarsten and angela97lin March 30, 2021 17:43

jeremyliweishih merged commit d6d2d84 into main Mar 31, 2021

freddyaboulton deleted the js_1999_dtf_nan branch May 13, 2022 15:01

		error_contains_nan = "Input datetime column ({}) contains NaN values. Please input NaN values or drop this column."


		class DateTimeNaNDataCheck(DataCheck):

Add DateTimeNaNDataCheck #2039

Add DateTimeNaNDataCheck #2039

Uh oh!

Conversation

jeremyliweishih commented Mar 25, 2021

Uh oh!

codecov bot commented Mar 25, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jeremyliweishih Mar 25, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bchen1116 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

bchen1116 commented Mar 26, 2021

Uh oh!

CLAassistant commented Mar 26, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ParthivNaresh left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

freddyaboulton left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

chukarsten left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

angela97lin left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Add `DateTimeNaNDataCheck` #2039

Add `DateTimeNaNDataCheck` #2039

codecov bot commented Mar 25, 2021 •

edited

Loading

jeremyliweishih Mar 25, 2021 •

edited

Loading

CLAassistant commented Mar 26, 2021 •

edited

Loading