STL Decomposer #3741

chukarsten · 2022-10-04T14:57:38Z

Decompose things...STL'ly!

codecov · 2022-10-04T15:09:21Z

Codecov Report

Merging #3741 (061f02e) into main (e7c35a5) will decrease coverage by 0.1%.
The diff coverage is 99.9%.

@@           Coverage Diff           @@
##            main   #3741     +/-   ##
=======================================
- Coverage   99.7%   99.7%   -0.0%     
=======================================
  Files        341     343      +2     
  Lines      35287   35681    +394     
=======================================
+ Hits       35155   35545    +390     
- Misses       132     136      +4

Impacted Files	Coverage Δ
evalml/pipelines/components/__init__.py	`100.0% <ø> (ø)`
...alml/pipelines/components/transformers/__init__.py	`100.0% <ø> (ø)`
...sts/decomposer_tests/test_polynomial_decomposer.py	`100.0% <ø> (ø)`
evalml/tests/component_tests/test_utils.py	`99.1% <ø> (ø)`
...omponent_tests/decomposer_tests/test_decomposer.py	`99.5% <99.5%> (-0.5%)`	⬇️
.../components/transformers/preprocessing/__init__.py	`100.0% <100.0%> (ø)`
...omponents/transformers/preprocessing/decomposer.py	`99.3% <100.0%> (-0.7%)`	⬇️
...ransformers/preprocessing/polynomial_decomposer.py	`100.0% <100.0%> (ø)`
...nents/transformers/preprocessing/stl_decomposer.py	`100.0% <100.0%> (ø)`
...nent_tests/decomposer_tests/test_stl_decomposer.py	`100.0% <100.0%> (ø)`
... and 5 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

…different classes. Moved the test to project the seasonal signal up to the parent Decomposer class. Moved the testing for the seasonal projection to the decomposer test module.

…composer.

… class.

… to move that up to the base Decomposer class.

* Updated get_trend_df() to work out of sample. * Fixed transform() to work with in sample, but not spanning the sample. * Fixed inverse_transform to work with smaller than sample, in sample data.

…ple. Also updated test for transform to return same if y is None and moved that to parent class.

…for fit.

…_period.

…etter reflect what's going on. Docstring changes.

…e seasonal sample to match the STLDecomposer.

… bug in forecasting the trend

…eriodicity.

…ynomialDecomposer.

eccabay

Fantastic 🚢 What beasts of tests these are.

evalml/pipelines/components/transformers/preprocessing/polynomial_decomposer.py

evalml/pipelines/components/transformers/preprocessing/stl_decomposer.py

eccabay · 2022-10-21T20:07:29Z

evalml/pipelines/components/transformers/preprocessing/stl_decomposer.py

+        if not isinstance(X.index, pd.DatetimeIndex):
+            raise TypeError("Provided X should have datetimes in the index.")


Do we need to continue enforcing this? We can set it within this function relatively easily.

evalml/tests/component_tests/decomposer_tests/test_decomposer.py

eccabay · 2022-10-21T20:26:40Z

evalml/tests/component_tests/decomposer_tests/test_decomposer.py

+    dec.set_seasonal_period(X, y)
+
+    assert 0.95 * period <= dec.seasonal_period <= 1.05 * period
+    assert dec.parameters["seasonal_period"]


Is this purposely just checking that the parameter exists? Should we be checking its value?

OK, yes, this one is a bug. For some reason, parameters dict value is not being set. I tried fixing it, but for some reason cannot get it to set. I filed this to look at it later given that we don't even use set_seasonal_period.

evalml/tests/component_tests/decomposer_tests/test_polynomial_decomposer.py

eccabay · 2022-10-21T20:31:44Z

evalml/tests/component_tests/decomposer_tests/test_stl_decomposer.py

+    assert stl.seasonal_period == 5
+
+
+def build_test_target(subset_y, seasonal_period, transformer_fit_on_data, to_test):


Worth it to move to conftest?

It is! This is actually handled in this parity PR and will shift up to the test_decomposer.py class.

evalml/tests/component_tests/decomposer_tests/test_stl_decomposer.py

jeremyliweishih

LGTM - amazing effort!

* cap featuretools * RL * Cap nlp primitives as well * Fix conda

…_period.

ParthivNaresh · 2022-10-25T14:38:43Z

I got the chance to play around with it and I think it looks and works great, fantastic job!

chukarsten · 2022-10-25T15:26:32Z

praise god

chukarsten force-pushed the stl_decomposer branch from 43ba31b to e38c45e Compare October 10, 2022 20:24

chukarsten and others added 25 commits October 11, 2022 10:42

fit and transform are working. inverse_transform in sample works.

f7682bf

Got inverse_transform tests working.

705e846

Changed the inverse_transform() variables to match each other in the …

45f1c16

…different classes. Moved the test to project the seasonal signal up to the parent Decomposer class. Moved the testing for the seasonal projection to the decomposer test module.

Added basic get_trend_df() to STLDecomposer. Moved testing up into De…

3b5ba47

…composer.

Moved the .set_period() tests up to the parent class.

f154e53

Moved .determine_periodicity() tests and functionality up to the base…

5f16e32

… class.

Changed the way we detect whether Decomposer has been fit. Still need…

0c7c860

… to move that up to the base Decomposer class.

Transform out of sample should work.

00ecf31

Transform handles partially in and out of sample data

f86eae8

Stl decomposer kc (#3747)

08e04c2

* Updated get_trend_df() to work out of sample. * Fixed transform() to work with in sample, but not spanning the sample. * Fixed inverse_transform to work with smaller than sample, in sample data.

Updated STL get_trend_dataframe() tests to work on less than full sam…

81fdf79

…ple. Also updated test for transform to return same if y is None and moved that to parent class.

Refactored Decomposer tests to all iterate across the two decomposers.

ca84050

I forget

8b25a2c

Removed unused test parameterization.

e4c127e

Moved plot function up to parent class.

6dc0af9

Updated base_meta to handle the new get_trend_df() function to check …

e677134

…for fit.

Docstring for init, release notes, test for auto changing of seasonal…

de0defd

…_period.

Changed Decomposer's _build_seasonal_signal to _project_seasonal to b…

e494503

…etter reflect what's going on. Docstring changes.

Refactored the checking for y to be None into the base class.

519d887

Moved .fit_transform() up to parent class.

041fa4a

Adjusted the PolynomialDecomposer to also save the seasonality and th…

e95646c

…e seasonal sample to match the STLDecomposer.

Backed out the check_target changes.

8933ecb

Docstring adjustments for inverse_transform.

d725bfb

Removed commented code.

4c1ff71

Boiler plate tests for test_components and test_utils.

a13b547

chukarsten force-pushed the stl_decomposer branch from b7c65ef to a13b547 Compare October 11, 2022 14:42

chukarsten added 2 commits October 11, 2022 10:48

Added the type annotations import to decomposer.py

283492e

Lint and swapped seasonal&seasonality.

c58798e

chukarsten and others added 14 commits October 20, 2022 10:57

Lint

f3c3e4a

Removed raise e

b2a8815

Made inverse_transform of STLDecomposer work with integer indices.

c77f18b

Made inverse_transform test run with integer index and fixed probable…

5b26447

… bug in forecasting the trend

Removed automl local test again.

fcc980a

Test coverage

020d7bd

Comments and removed unnecessary TODO.

ec0a816

Addressed Becca's comments.

fa30e42

Fixed index test that somehow got broken.

40325fc

Typos.

383390f

Update stl seasonal period after fitting

e7cfbf0

Update index finding in _project_seasonal

dcb4db0

Updated becca's change to transform_first_ind to be general for the p…

fbd1c4e

…eriodicity.

Addressed Parthiv's comment to add in the original index into the Pol…

c271044

…ynomialDecomposer.

eccabay approved these changes Oct 21, 2022

View reviewed changes

eccabay and others added 3 commits October 24, 2022 08:57

Save last seasonal period instead of first

dbaf70c

Merge branch 'main' into stl_decomposer

b5b3cc1

Becca's comments addressed.

7dc1eae

jeremyliweishih approved these changes Oct 24, 2022

View reviewed changes

jeremyliweishih and others added 5 commits October 25, 2022 00:50

Cap Featuretools at < 1.15.0 (#3775)

06a207f

* cap featuretools * RL * Cap nlp primitives as well * Fix conda

Docstring for init, release notes, test for auto changing of seasonal…

05f1ad1

…_period.

Empty-Commit

a1b4c47

Merge branch 'main' into stl_decomposer

2cbadc6

Removed additional test.

9186ba6

Merge branch 'main' into stl_decomposer

061f02e

chukarsten merged commit fa71b7b into main Oct 25, 2022

chukarsten deleted the stl_decomposer branch October 25, 2022 15:26

chukarsten mentioned this pull request Oct 26, 2022

Release v0.61.0. #3782

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

STL Decomposer #3741

STL Decomposer #3741

chukarsten commented Oct 4, 2022

codecov bot commented Oct 4, 2022 •

edited

Loading

eccabay left a comment

eccabay Oct 21, 2022

eccabay Oct 21, 2022

chukarsten Oct 24, 2022

eccabay Oct 21, 2022

chukarsten Oct 24, 2022

jeremyliweishih left a comment

ParthivNaresh commented Oct 25, 2022

chukarsten commented Oct 25, 2022

		if not isinstance(X.index, pd.DatetimeIndex):
		raise TypeError("Provided X should have datetimes in the index.")

		assert stl.seasonal_period == 5


		def build_test_target(subset_y, seasonal_period, transformer_fit_on_data, to_test):

STL Decomposer #3741

STL Decomposer #3741

Conversation

chukarsten commented Oct 4, 2022

codecov bot commented Oct 4, 2022 • edited Loading

Codecov Report

eccabay left a comment

Choose a reason for hiding this comment

eccabay Oct 21, 2022

Choose a reason for hiding this comment

eccabay Oct 21, 2022

Choose a reason for hiding this comment

chukarsten Oct 24, 2022

Choose a reason for hiding this comment

eccabay Oct 21, 2022

Choose a reason for hiding this comment

chukarsten Oct 24, 2022

Choose a reason for hiding this comment

jeremyliweishih left a comment

Choose a reason for hiding this comment

ParthivNaresh commented Oct 25, 2022

chukarsten commented Oct 25, 2022

codecov bot commented Oct 4, 2022 •

edited

Loading