Statically set woodwork typing in tests #3697
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…licitly set types
Codecov Report
@@ Coverage Diff @@
## main #3697 +/- ##
=======================================
- Coverage 99.7% 99.7% -0.0%
=======================================
Files 339 339
Lines 34465 34386 -79
=======================================
- Hits 34338 34254 -84
- Misses 127 132 +5
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here. |
eccabay
requested review from
ParthivNaresh,
jeremyliweishih,
bchen1116,
christopherbunn,
chukarsten and
fjlanasa
and removed request for
ParthivNaresh
gsheni
reviewed
Sep 9, 2022
evalml/pipelines/components/transformers/imputers/time_series_imputer.py
Outdated
Show resolved
Hide resolved
|
Moving back to draft |
eccabay
commented
Sep 13, 2022
Comment on lines
-261
to
+263
| logical_types={col: "Double" for col in cols_derived_from_categoricals}, | ||
| logical_types={col: "Double" for col in lagged_features.columns}, |
There was a problem hiding this comment.
I'm not confident in the correctness of this change, as to how we handle all types and if everything does in fact become a double here. If anyone knows better about this, please let me know.
chukarsten
approved these changes
Sep 15, 2022
Comment on lines
-4549
to
-4565
| if problem_type == ProblemTypes.TIME_SERIES_MULTICLASS: | ||
| X, y = ts_data_multi | ||
| elif problem_type == ProblemTypes.TIME_SERIES_BINARY: | ||
| X, y = ts_data_binary | ||
| else: | ||
| X, y = ts_data |
| @@ -568,7 +566,7 @@ def test_simple_imputer_ignores_natural_language( | |||
| ans = X_df.mode().iloc[0, :] | |||
| ans["natural language col"] = pd.NA | |||
| X_df.iloc[-1, :] = ans | |||
| assert_frame_equal(result, X_df) | |||
| assert_frame_equal(result, X_df, check_dtype=False) | |||
jeremyliweishih
approved these changes
Sep 15, 2022
| @@ -297,7 +297,22 @@ def test_schema_is_equal_fraud(fraud_100): | |||
| assert _schema_is_equal(X.ww.schema, X2.ww.schema) | |||
|
|
|||
|
|
|||
| def test_test_downcast_nullable_types_can_handle_no_schema(): | |||
| def test_downcast_nullable_types_series(): | |||
There was a problem hiding this comment.
do you want to add the test case with NaNs as well?
|
|
||
| X_bool_nullable_cols = X.ww.select("BooleanNullable") | ||
| X_int_nullable_cols = X.ww.select(["IntegerNullable", "AgeNullable"]) | ||
| if isinstance(data, pd.Series): |
There was a problem hiding this comment.
if the series contains null values should we still downcast into Boolean or Double? I think we should follow what we have and use ignore_null_cols to differentiate.
Merged
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #3651
This is not a woodwork 0.17.2 upgrade. The goal was simply to add explicit woodwork typing throughout the tests to make them more resilient to woodwork inference changes. The end result of this does have fewer tests failing in 0.17.2 since I used the upgrade as a check for general stability.
This does include some setting of woodwork types in components though, related to my previous work about being explicit about woodwork typing to reduce the number of times we re-infer within components (plus it makes tests pass 😁).
Changes in this PR:
ts_dataandget_ts_X_ytest pytest fixtures into one fixture, namedts_databut with data fromget_ts_X_y.ts_data,imputer_test_data,X_y_binary,X_y_multi,X_y_regression,X_y_categorical_classification, andX_y_categorical_regressionto explicitly set woodwork types.downcast_nullable_typesnow works with either a DataFrame or a Series as input data (instead of just dataframes), and there's now a test for it.ww.init()ininfer_feature_typesin the case where we already have a valid schema for the input data. This has performance implications and there are performance test results for this branch.Before vs after woodwork upgrade comparison:
Main/0.16.4 -> 0 failing tests
Main/0.17.2 -> 287 failing tests
Main /0.18.0 -> 291 failing tests
Branch/0.16.4 -> 0 failing tests
Branch/0.17.2 -> 23 failing tests
Branch/0.18.0 -> 22 failing tests
I apologize in advance for the length of this PR 😬