Automatic Periodicty Determination #3729

chukarsten · 2022-09-22T05:38:58Z

In order to better calculate the seasonal signal, the Decomposer child objects should be updated with a function to estimate the period of the seasonal component of that signal. This PR adds a function to estimate the period using either an autocorrelation or partial autocorrelation based method. Currently, the autocorrelation method works best. Follow-on stories are filed to continue work on these functions.

…ile for tests.

…ne period.

codecov · 2022-09-22T05:45:58Z

Codecov Report

Merging #3729 (6fa0157) into main (7a995be) will increase coverage by 0.1%.
The diff coverage is 100.0%.

@@           Coverage Diff           @@
##            main   #3729     +/-   ##
=======================================
+ Coverage   99.7%   99.7%   +0.1%     
=======================================
  Files        338     339      +1     
  Lines      34672   34786    +114     
=======================================
+ Hits       34540   34655    +115     
+ Misses       132     131      -1

Impacted Files	Coverage Δ
...ransformers/preprocessing/polynomial_decomposer.py	`100.0% <ø> (+0.8%)`	⬆️
...omponents/transformers/preprocessing/decomposer.py	`100.0% <100.0%> (ø)`
...omponent_tests/decomposer_tests/test_decomposer.py	`100.0% <100.0%> (ø)`
...sts/decomposer_tests/test_polynomial_decomposer.py	`100.0% <100.0%> (ø)`
evalml/tests/conftest.py	`98.1% <100.0%> (+0.1%)`	⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

…r of the relative extrema, filtering out negative ACF values and just grabbing the max.

chukarsten · 2022-09-27T04:13:53Z

evalml/pipelines/components/transformers/preprocessing/decomposer.py

@@ -26,10 +32,122 @@ def __init__(self, parameters=None, component_obj=None, random_seed=0, **kwargs)
            **kwargs,
        )

+    def _set_time_index(self, X, y):


Shifted _set_time_index() up from PolynomialDecomposer class to the parent since it can probably be reused.

eccabay

Awesome to see this, and I'm glad the implementation isn't overly-complicated. My questions are mainly about the intended flow within the decomposer. When are the periodic and detrender functions expected to be called within the flow fitting and predicting on a pipeline? This might be buried in the code somewhere but it's currently unclear to me.

eccabay · 2022-09-27T13:55:44Z

docs/source/user_guide/timeseries.ipynb

+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "The children of the `Decomposer` class, if not set in the constructor, sets its `seasonal_period` parameter to -1.  If the `Decomposer` object is fit with a `seasonal_period` of -1, it will take on a default value of 7, which is good for daily data signals that have a known seasonal component period that is weekly.  \n",


I'm a bit confused by this. The default is -1, but actually 7? Why have the separate values and not just default to 7 immediately?

Another note - why leave the seasonal_period to the children? Seems like it would fit better in the parent Decomposer class, right?

With respect to leaving the seasonal_period to the children, I initially did that because I was not entirely sure whether the other children of Decomposer would need this functionality or not. I filed this issue to come back to it after we do one or two more Decomposer children.

With respect to the "default" value, I understand the confusion. I changed the wording in the notebook as the default value is not really even 7. "-1" means "let statsmodels' freq_to_period figure it out base on the inferred frequency of the datetime index." Setting any other integer makes that the seasonal period.

eccabay · 2022-09-27T13:57:08Z

docs/source/user_guide/timeseries.ipynb

+   "source": [
+    "The children of the `Decomposer` class, if not set in the constructor, sets its `seasonal_period` parameter to -1.  If the `Decomposer` object is fit with a `seasonal_period` of -1, it will take on a default value of 7, which is good for daily data signals that have a known seasonal component period that is weekly.  \n",
+    "\n",
+    "However, if the seasonal period is not known before hand, the `set_seasonal_period()` convenience function will look at the target data, determine the period and set the `Decomposer` object appropriately."


This sentence seems to conflict with the above. Can you clarify the flow of how the parameter gets set if the user does not set it themselves?

Unrelated nit: "before hand" -> "beforehand"

What do you think of this in the cell:

The children of the `Decomposer` class, if not explicitly set in the constructor, set their `seasonal_period` parameter based on a `statsmodels` function, `freq_to_period`, that considers the frequency of the datetime data. If the `PolynomialDecomposer` object is fit with `seasonal_period` not explicitly set, it will take on a default value of 7, which is good for daily data signals that have a known seasonal component period that is weekly. However, if the seasonal period is not known beforehand, the `set_seasonal_period()` convenience function will look at the target data, determine a better guess for the period and set the `Decomposer` object appropriately.

That makes a bit more sense. I'm still slightly confused though, does the set_seasonal_period() function replace the freq_to_period function with a better guess, or are they completely unrelated?

set_seasonal_period() does indeed set seasonal_period with a better guess. freq_to_period basically just looks at the data and says "oh, the granularity of the time series is days, let's use 7." and has a similar mapping for other frequencies. set_seasonal_period() will actually invoke the seasonal period detection to make a good guess, then set it.

eccabay · 2022-09-27T13:58:36Z

docs/source/user_guide/timeseries.ipynb

+    "pdc.fit(X_train_time, y_train_time)\n",
+    "assert pdc.seasonal_period == 7\n",
+    "pdc.set_seasonal_period(X_train_time, y_train_time)\n",
+    "assert 363 < pdc.seasonal_period < 368"


Why this range? Would the seasonal period ever be something different than 365 or 366? (And if so, does it make sense to fix it since the data in this example is known?)

I was just trying to show the "correct" period of ~365 +- a little fuzziness due to the algorithm.

eccabay · 2022-09-27T14:08:09Z

evalml/pipelines/components/transformers/preprocessing/decomposer.py

+            pacf = sm.tsa.pacf(y)
+            return argrelextrema(pacf, np.greater)[0]
+
+        def _detrend_on_fly(X, y):


I'm worried about this - these are lots of computationally expensive calls. Remind me, when in the process do we expect to be using the periodicity?

If I understand correctly, we need the period to properly detrend, so this is a pre-detrending to get the period? What a fun catch-22... If this is the case, can we reduce this in any way by enforcing this is run after either the component is fit or the time index is already set?

Sadly, it cannot be run after the component is fit...as the intent is to use the output to fit the component properly. This does seem a bit like a catch-22 because I think really this is kind of an iterative process. I am not sure what you mean though for the time index, though.

Gotcha. My thinking with the time index is that set_time_index can do a lot of extra work if it's run multiple times - ideally there'd be some way we can make sure its called as few times as possible within the component.

evalml/pipelines/components/transformers/preprocessing/decomposer.py

evalml/tests/component_tests/decomposer_tests/test_polynomial_decomposer.py

eccabay · 2022-09-27T14:16:07Z

evalml/tests/component_tests/decomposer_tests/test_polynomial_decomposer.py

+        X, y = load_weather()
+        y = y.set_axis(X["Date"]).asfreq(pd.infer_freq(X["Date"]))
+        X = X.set_index("Date").asfreq(pd.infer_freq(X["Date"]))
+    return X, y


Without looking at this closely, is there any value in initializing woodwork here?

…o fixture.

chukarsten added 8 commits September 20, 2022 14:14

Release v0.58.0.

289c441

Initial shot at adding a .determine_periodicity() function.

81f1347

Moved _set_time_axis() up to parent Decomposer class and added stub f…

0b7f266

…ile for tests.

Added detrend on the fly capability to enhance ACF ability to determi…

d96f629

…ne period.

Cleaned up code a little bit.

2ba1522

Added the test for _set_time_index

e78c5e7

Updated doc string and fixed it to make sure the parameters changed too.

cf74f38

Removed extraneous code.

2b5874f

auto-assign bot assigned chukarsten Sep 22, 2022

chukarsten and others added 2 commits September 22, 2022 01:42

Release notes.

211aa7a

Merge branch 'main' into determine_periodicity

8a2c404

chukarsten and others added 14 commits September 22, 2022 18:05

Got weather dataset working with the synthetics by massaging the orde…

9cacdee

…r of the relative extrema, filtering out negative ACF values and just grabbing the max.

Docstring change.

07324cf

Notebook.

49b3721

Loosened up the tests a little.

0d1c519

Changed out the evalml_kernel switch.

0221f60

EXPERIMENTAL.

5d04f50

Merge branch 'main' into determine_periodicity

2b60f07

Lint.

b982784

Catboost

b7d461b

Updated docs.

9bf4092

Update .gitignore

8424355

Update notebook_version_standardizer.py

cc9d2ac

Update release_notes.rst

5ead5bf

Update decomposer.py

c96bf32

chukarsten commented Sep 27, 2022

View reviewed changes

chukarsten changed the title ~~Determine periodicity~~ Automatic Periodicty Determination Sep 27, 2022

chukarsten and others added 2 commits September 27, 2022 00:29

Lint.

60c9f42

Merge branch 'main' into determine_periodicity

c770ee2

chukarsten requested review from eccabay, jeremyliweishih, ParthivNaresh, christopherbunn, dsherry, fjlanasa and rpeck September 27, 2022 04:32

eccabay requested changes Sep 27, 2022

View reviewed changes

chukarsten and others added 6 commits September 27, 2022 15:36

ADdressed some of Becca's comments and moved generate_seasonal_data t…

b0fa400

…o fixture.

Addressed more comments and adjusted the ipynb cell.

abee371

Merge branch 'main' into determine_periodicity

51f4579

Merge branch 'main' into determine_periodicity

86e43f6

Lint.

64d86ad

Updated ipynb again.

6fa0157

chukarsten merged commit ae43681 into main Sep 27, 2022

chukarsten deleted the determine_periodicity branch September 27, 2022 23:45

chukarsten mentioned this pull request Sep 27, 2022

Release v0.59.0 #3735

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Automatic Periodicty Determination #3729

Automatic Periodicty Determination #3729

chukarsten commented Sep 22, 2022 •

edited

Loading

codecov bot commented Sep 22, 2022 •

edited

Loading

chukarsten Sep 27, 2022

eccabay left a comment

eccabay Sep 27, 2022

eccabay Sep 27, 2022

chukarsten Sep 27, 2022

chukarsten Sep 27, 2022

eccabay Sep 27, 2022

chukarsten Sep 27, 2022

eccabay Sep 27, 2022

chukarsten Sep 27, 2022

eccabay Sep 27, 2022

chukarsten Sep 27, 2022

eccabay Sep 27, 2022

chukarsten Sep 27, 2022

eccabay Sep 27, 2022

eccabay Sep 27, 2022

Automatic Periodicty Determination #3729

Automatic Periodicty Determination #3729

Conversation

chukarsten commented Sep 22, 2022 • edited Loading

codecov bot commented Sep 22, 2022 • edited Loading

Codecov Report

Choose a reason for hiding this comment

eccabay left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

chukarsten commented Sep 22, 2022 •

edited

Loading

codecov bot commented Sep 22, 2022 •

edited

Loading