Add Exponential Smoothing time series estimator by eccabay · Pull Request #3157 · alteryx/evalml

eccabay · 2021-12-16T19:21:26Z

Closes #2991. This will add the estimator to AutoML, as per the current perf test results.

codecov · 2021-12-16T19:26:15Z

Codecov Report

Merging #3157 (574be6e) into main (47d4867) will increase coverage by 0.1%.
The diff coverage is 100.0%.

@@           Coverage Diff           @@
##            main   #3157     +/-   ##
=======================================
+ Coverage   99.7%   99.7%   +0.1%     
=======================================
  Files        324     326      +2     
  Lines      31302   31432    +130     
=======================================
+ Hits       31198   31328    +130     
  Misses       104     104

Impacted Files	Coverage Δ
evalml/pipelines/__init__.py	`100.0% <ø> (ø)`
evalml/pipelines/components/__init__.py	`100.0% <ø> (ø)`
evalml/pipelines/components/estimators/__init__.py	`100.0% <ø> (ø)`
evalml/tests/component_tests/test_utils.py	`95.7% <ø> (ø)`
...alml/tests/model_family_tests/test_model_family.py	`100.0% <ø> (ø)`
evalml/model_family/model_family.py	`100.0% <100.0%> (ø)`
...lines/components/estimators/regressors/__init__.py	`100.0% <100.0%> (ø)`
...tors/regressors/exponential_smoothing_regressor.py	`100.0% <100.0%> (ø)`
...alml/tests/component_tests/test_arima_regressor.py	`100.0% <100.0%> (ø)`
evalml/tests/component_tests/test_components.py	`99.3% <100.0%> (ø)`
... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 47d4867...574be6e. Read the comment docs.

ParthivNaresh · 2021-12-17T17:24:02Z

evalml/pipelines/components/estimators/regressors/exponential_smoothing_regressor.py

+            random_seed=random_seed,
+        )
+
+    def _remove_datetime(self, data, features=False):


Nit: Since _remove_datetime, _match_indices, and _set_forecast don't rely on the class instance and are shared by ARIMARegressor (and potentially future similar estimators), you can move these to pipelines/components/utils. Not necessary for this PR and we can file a separate issue for code cleanup if necessary.

evalml/tests/component_tests/test_exponential_smoothing_regressor.py

bchen1116

Great work here so far! There's some errors in the docstrings, and I believe a test should be fixed/updated for better coverage, but it's looking good!

evalml/model_family/model_family.py

evalml/pipelines/components/estimators/regressors/exponential_smoothing_regressor.py

evalml/tests/component_tests/test_exponential_smoothing_regressor.py

evalml/pipelines/components/estimators/regressors/exponential_smoothing_regressor.py

bchen1116

Left some comments, but exciting to be able to add this in soon!

evalml/pipelines/components/estimators/regressors/exponential_smoothing_regressor.py

evalml/tests/component_tests/test_exponential_smoothing_regressor.py

bchen1116

Thanks for making the changes! This looks great to me. Left one testing change suggestion, but not blocking!

evalml/tests/component_tests/test_exponential_smoothing_regressor.py

freddyaboulton

@eccabay Implementation looks good to me! Thanks for making the changes. Waiting on the latest batch of perf tests.

evalml/pipelines/components/estimators/regressors/exponential_smoothing_regressor.py

freddyaboulton · 2022-01-03T22:15:36Z

evalml/pipelines/components/estimators/regressors/exponential_smoothing_regressor.py

+        trend=None,
+        damped_trend=False,
+        seasonal=None,
+        sp=2,


Given that the default value of seasonal is None, shouldn't the default value of sp be None too? Is sp used only if seasonal is not None?

I would have thought so as well, but there's something in the sktime implementation that enforces otherwise. I can't tell exactly where sp is used when seasonal is None, but it fails otherwise.

chukarsten

A few nits and questions, but nothing blocking. Looks great! Good work!

chukarsten · 2022-01-05T16:05:53Z

evalml/pipelines/components/estimators/regressors/exponential_smoothing_regressor.py

+        return data_no_dt
+
+    def _set_forecast(self, X):
+        from sktime.forecasting.base import ForecastingHorizon


Are we importing here to allow the import_or_raise to go first?

chukarsten · 2022-01-05T16:14:31Z

evalml/tests/component_tests/test_arima_regressor.py

        clf.fit(X=X)


-@pytest.fixture


Very nice, love this fixture reusage and combination.

chukarsten · 2022-01-05T16:28:11Z

evalml/tests/component_tests/test_exponential_smoothing_regressor.py

+        assert not isinstance(y_train_removed.index, pd.DatetimeIndex)
+
+
+def test_set_forecast(get_ts_X_y):


It seems kind of weird to be testing what's going on purely internally to the regressor!

chukarsten · 2022-01-05T16:48:56Z

evalml/tests/component_tests/test_exponential_smoothing_regressor.py

+    clf = ExponentialSmoothingRegressor()
+    with patch.object(clf, "_component_obj"):
+        clf.fit(X, y)
+        assert clf.feature_importance == np.zeros(1)


Filing this to standardize the way we handle these not-supported feature importance tests.

evalml/tests/component_tests/test_exponential_smoothing_regressor.py

eccabay added 5 commits December 8, 2021 15:07

Init

ab67002

Merge branch 'main' into 2991_exponential_smoothing

bcd6a5e

Some testing

006f3f7

Merge branch 'main' into 2991_exponential_smoothing

9eb76c8

More tests

61fee2f

eccabay added 2 commits December 16, 2021 15:12

Update release notes

1c75a9f

Test fix

95f09c2

eccabay self-assigned this Dec 16, 2021

eccabay added 3 commits December 17, 2021 08:48

more test fixing

67476b0

Typo fixes

8674406

Merge branch 'main' into 2991_exponential_smoothing

9634c28

eccabay marked this pull request as ready for review December 17, 2021 16:40

eccabay requested review from ParthivNaresh, angela97lin, bchen1116, chukarsten and freddyaboulton December 17, 2021 16:40

ParthivNaresh reviewed Dec 17, 2021

View reviewed changes

bchen1116 requested changes Dec 17, 2021

View reviewed changes

eccabay added 2 commits December 17, 2021 14:56

Move get_X_y to conftest fixture

098d6d8

Address PR comments

7b0b3d4

ParthivNaresh reviewed Dec 20, 2021

View reviewed changes

evalml/pipelines/components/estimators/regressors/exponential_smoothing_regressor.py Show resolved Hide resolved

Expose hyperparameters

c36b3ba

freddyaboulton reviewed Dec 20, 2021

View reviewed changes

evalml/pipelines/components/estimators/regressors/exponential_smoothing_regressor.py Outdated Show resolved Hide resolved

eccabay added 4 commits December 21, 2021 09:31

Add ExponentialSmoothing to automl for perf tests (oops)

7205a1b

Update parameters to actually work

88eb055

Remove X transformations

16c42ad

Merge branch 'main' into 2991_exponential_smoothing

74349b3

eccabay requested a review from bchen1116 December 22, 2021 16:30

bchen1116 reviewed Dec 22, 2021

View reviewed changes

more PR comments

cfa0073

eccabay requested a review from bchen1116 December 22, 2021 21:37

bchen1116 approved these changes Dec 23, 2021

View reviewed changes

evalml/tests/component_tests/test_exponential_smoothing_regressor.py Show resolved Hide resolved

eccabay added 4 commits January 3, 2022 08:35

Merge branch 'main' into 2991_exponential_smoothing

ffe526c

Small hyperparameter correctness updates

c0d9f1f

Increase min sp

bdaf416

remove buggy use_boxcox

8b83ec7

freddyaboulton reviewed Jan 3, 2022

View reviewed changes

eccabay added 3 commits January 4, 2022 09:35

more PR fixes

256abe9

fix failing test?

2dc9857

codecov fix

50ad0f7

chukarsten approved these changes Jan 5, 2022

View reviewed changes

eccabay added 5 commits January 5, 2022 13:42

Remove multiplicative from trend and seasonal hyperparameters

d051fa9

testing cleanup

dfdec93

Merge branch 'main' into 2991_exponential_smoothing

c746ab1

lint

e1441f7

Merge branch 'main' into 2991_exponential_smoothing

574be6e

eccabay merged commit 3ad0d3c into main Jan 5, 2022

eccabay deleted the 2991_exponential_smoothing branch March 10, 2022 15:34

		assert not isinstance(y_train_removed.index, pd.DatetimeIndex)


		def test_set_forecast(get_ts_X_y):

		clf.fit(X=X)


		@pytest.fixture

Conversation

eccabay commented Dec 16, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Dec 16, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

ParthivNaresh Dec 17, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

bchen1116 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bchen1116 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bchen1116 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

freddyaboulton left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

freddyaboulton Jan 3, 2022

Choose a reason for hiding this comment

Uh oh!

eccabay Jan 4, 2022

Choose a reason for hiding this comment

Uh oh!

chukarsten left a comment

Choose a reason for hiding this comment

Uh oh!

chukarsten Jan 5, 2022

Choose a reason for hiding this comment

Uh oh!

chukarsten Jan 5, 2022

Choose a reason for hiding this comment

Uh oh!

chukarsten Jan 5, 2022

Choose a reason for hiding this comment

Uh oh!

chukarsten Jan 5, 2022

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

eccabay commented Dec 16, 2021 •

edited

Loading

codecov bot commented Dec 16, 2021 •

edited

Loading