Integrate Decomposition into AutoMLSearch #3781

eccabay · 2022-10-25T17:29:27Z

Includes some other small code changes to allow integration to be successful!

…different classes. Moved the test to project the seasonal signal up to the parent Decomposer class. Moved the testing for the seasonal projection to the decomposer test module.

…composer.

… class.

… to move that up to the base Decomposer class.

* Updated get_trend_df() to work out of sample. * Fixed transform() to work with in sample, but not spanning the sample. * Fixed inverse_transform to work with smaller than sample, in sample data.

…ple. Also updated test for transform to return same if y is None and moved that to parent class.

…for fit.

…_period.

…etter reflect what's going on. Docstring changes.

…e seasonal sample to match the STLDecomposer.

…s and regression pipelines.

eccabay · 2022-10-25T17:33:28Z

evalml/pipelines/time_series_pipeline_base.py

@@ -116,6 +117,16 @@ def _add_training_data_to_X_Y(self, X, y, X_train, y_train):
                self.gap,
            )

+            # Properly fill in the dates in the gap


This was a pre-existing bug where we fill in the gap between the training and testing data with the last row of training data, but that means creating an irregularly spaced time index.

eccabay · 2022-10-25T17:35:36Z

evalml/pipelines/time_series_pipeline_base.py

+            X_schema = X.ww.schema
+            y_schema = y.ww.schema
+            X = X.set_index(X[self.time_index])
+            y = y.set_axis(X[self.time_index])
+            X.ww.init(schema=X_schema)
+            y.ww.init(schema=y_schema)


Feels weird to be saving the time index in the data's indices when we're trying to drop it, but the decomposer needs the time index data to decompose successfully. If anyone has suggestions on a better or cleaner way to get around this I am very open to it.

Can we drop after fit or transform in the decomposer then? The problem is non-time-series native estimators will fail with a datetime column. Maybe we can amend the logic here to save the time index only when having the decomposer in the pipeline and then dropping the time index later?

codecov · 2022-10-25T17:36:06Z

Codecov Report

Merging #3781 (6f4f2de) into main (ed2a248) will decrease coverage by 0.1%.
The diff coverage is 100.0%.

@@           Coverage Diff           @@
##            main   #3781     +/-   ##
=======================================
- Coverage   99.7%   99.7%   -0.0%     
=======================================
  Files        343     343             
  Lines      35680   35754     +74     
=======================================
+ Hits       35544   35617     +73     
- Misses       136     137      +1

Impacted Files	Coverage Δ
.../integration_tests/test_time_series_integration.py	`100.0% <ø> (ø)`
evalml/utils/__init__.py	`100.0% <ø> (ø)`
...omponents/transformers/preprocessing/decomposer.py	`99.1% <100.0%> (-0.1%)`	⬇️
...nents/transformers/preprocessing/stl_decomposer.py	`100.0% <100.0%> (ø)`
.../pipelines/time_series_classification_pipelines.py	`100.0% <100.0%> (ø)`
evalml/pipelines/time_series_pipeline_base.py	`99.2% <100.0%> (-0.8%)`	⬇️
...valml/pipelines/time_series_regression_pipeline.py	`100.0% <100.0%> (ø)`
evalml/pipelines/utils.py	`99.6% <100.0%> (+0.1%)`	⬆️
evalml/tests/automl_tests/test_automl.py	`99.5% <100.0%> (+0.1%)`	⬆️
evalml/tests/conftest.py	`98.0% <100.0%> (+0.1%)`	⬆️
... and 5 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

jeremyliweishih

I think we need to figure out when and where we're dropping the time index so that the pipelines don't fail for estimators that can't accept time indexes but everything else LGTM!

jeremyliweishih · 2022-10-25T19:45:11Z

evalml/pipelines/time_series_pipeline_base.py

+            X_schema = X.ww.schema
+            y_schema = y.ww.schema
+            X = X.set_index(X[self.time_index])
+            y = y.set_axis(X[self.time_index])
+            X.ww.init(schema=X_schema)
+            y.ww.init(schema=y_schema)


Can we drop after fit or transform in the decomposer then? The problem is non-time-series native estimators will fail with a datetime column. Maybe we can amend the logic here to save the time index only when having the decomposer in the pipeline and then dropping the time index later?

jeremyliweishih · 2022-10-25T19:48:27Z

can we also add test_automl_supports_time_series_regression to make sure that the decomposer and its parameters are correct?

jeremyliweishih

nvm - we're good but think we should add to test_automl_supports_time_series_regression still!

eccabay · 2022-10-26T19:07:56Z

Closing in favor of #3785

chukarsten and others added 30 commits October 11, 2022 10:42

fit and transform are working. inverse_transform in sample works.

f7682bf

Got inverse_transform tests working.

705e846

Changed the inverse_transform() variables to match each other in the …

45f1c16

…different classes. Moved the test to project the seasonal signal up to the parent Decomposer class. Moved the testing for the seasonal projection to the decomposer test module.

Added basic get_trend_df() to STLDecomposer. Moved testing up into De…

3b5ba47

…composer.

Moved the .set_period() tests up to the parent class.

f154e53

Moved .determine_periodicity() tests and functionality up to the base…

5f16e32

… class.

Changed the way we detect whether Decomposer has been fit. Still need…

0c7c860

… to move that up to the base Decomposer class.

Transform out of sample should work.

00ecf31

Transform handles partially in and out of sample data

f86eae8

Stl decomposer kc (#3747)

08e04c2

* Updated get_trend_df() to work out of sample. * Fixed transform() to work with in sample, but not spanning the sample. * Fixed inverse_transform to work with smaller than sample, in sample data.

Updated STL get_trend_dataframe() tests to work on less than full sam…

81fdf79

…ple. Also updated test for transform to return same if y is None and moved that to parent class.

Refactored Decomposer tests to all iterate across the two decomposers.

ca84050

I forget

8b25a2c

Removed unused test parameterization.

e4c127e

Moved plot function up to parent class.

6dc0af9

Updated base_meta to handle the new get_trend_df() function to check …

e677134

…for fit.

Docstring for init, release notes, test for auto changing of seasonal…

de0defd

…_period.

Changed Decomposer's _build_seasonal_signal to _project_seasonal to b…

e494503

…etter reflect what's going on. Docstring changes.

Refactored the checking for y to be None into the base class.

519d887

Moved .fit_transform() up to parent class.

041fa4a

Adjusted the PolynomialDecomposer to also save the seasonality and th…

e95646c

…e seasonal sample to match the STLDecomposer.

Backed out the check_target changes.

8933ecb

Docstring adjustments for inverse_transform.

d725bfb

Removed commented code.

4c1ff71

Boiler plate tests for test_components and test_utils.

a13b547

Added the type annotations import to decomposer.py

283492e

Lint and swapped seasonal&seasonality.

c58798e

Code cov.

fdd5878

Removed unused code.

ad5021b

Merge branch 'main' into stl_decomposer

5248c89

eccabay and others added 12 commits October 24, 2022 09:29

Remove erroneous asserts

bb8cd15

Move time index detection from decomposer to utils

20e51a3

Add frequency check to building pipelines

6c1b021

lint fixes

6a558a1

Fix circular import

e7b80c6

Fixed most of the tests by only applying the decomposer to time serie…

c348479

…s and regression pipelines.

Testing fixes

93083ea

New tests w/some test fixes

371b84f

Merge branch 'main' into stl_after_test

62982db

Fix codecov

f153e9b

Update release notes

9abf20c

Merge branch 'main' into stl_in_automl

22b6643

eccabay commented Oct 25, 2022

View reviewed changes

Lint fix

6f4f2de

eccabay marked this pull request as ready for review October 25, 2022 19:17

auto-assign bot assigned eccabay Oct 25, 2022

eccabay requested review from chukarsten, jeremyliweishih, christopherbunn, fjlanasa and ParthivNaresh October 25, 2022 19:18

jeremyliweishih requested changes Oct 25, 2022

View reviewed changes

jeremyliweishih approved these changes Oct 25, 2022

View reviewed changes

chukarsten mentioned this pull request Oct 26, 2022

Integration of STL into EvalML #3785

Merged

eccabay closed this Oct 26, 2022

eccabay deleted the stl_in_automl branch November 15, 2022 16:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Integrate Decomposition into AutoMLSearch #3781

Integrate Decomposition into AutoMLSearch #3781

eccabay commented Oct 25, 2022

eccabay Oct 25, 2022

eccabay Oct 25, 2022

jeremyliweishih Oct 25, 2022

codecov bot commented Oct 25, 2022 •

edited

Loading

jeremyliweishih left a comment •

edited

Loading

jeremyliweishih Oct 25, 2022

jeremyliweishih commented Oct 25, 2022

jeremyliweishih left a comment

eccabay commented Oct 26, 2022

Integrate Decomposition into AutoMLSearch #3781

Integrate Decomposition into AutoMLSearch #3781

Conversation

eccabay commented Oct 25, 2022

eccabay Oct 25, 2022

Choose a reason for hiding this comment

eccabay Oct 25, 2022

Choose a reason for hiding this comment

jeremyliweishih Oct 25, 2022

Choose a reason for hiding this comment

codecov bot commented Oct 25, 2022 • edited Loading

Codecov Report

jeremyliweishih left a comment • edited Loading

Choose a reason for hiding this comment

jeremyliweishih Oct 25, 2022

Choose a reason for hiding this comment

jeremyliweishih commented Oct 25, 2022

jeremyliweishih left a comment

Choose a reason for hiding this comment

eccabay commented Oct 26, 2022

codecov bot commented Oct 25, 2022 •

edited

Loading

jeremyliweishih left a comment •

edited

Loading