Re-design of Theta Forecaster

[Lovkush-A]

@mloning @fkiraly

To understand AutoTheta forecaster I read this paper: https://www.researchgate.net/publication/338507620_Generalizing_the_Theta_method_for_automatic_forecasting

In doing so, I do not think current implementation of Theta algorithm is fully correct. Here is my current understanding:

• We have time series Yt = y1,...,yn.
• If Yt is seasonal, then deseasonalise it, and reseasonalise the predictions at the end
• For any theta >= 0, you can calculate related series Zt(theta)
  • Zt(0) is the linear trend of Yt, i.e. the predictions when you apply linear regression with inputs as t and outputs as Yt
  • Zt(1) is just the original series Yt
  • Roughly, theta controls the 'distance' from a linear trend. Alternatively, theta roughly controls the curvature. Alternatively, large theta magnifies local variations and small theta puts more weight on global (linear) trend
• Pick theta_i for i=1,...,n_theta. Standard choice is n_theta=2
• Forecast Z(theta_i) somehow (e.g. exponential smoothing)
• Then let prediction of Yt be a weighted sum of predictions of the Z(theta_i)

The current implementation of Theta forecaster in sktime is a special case of this general process with n_theta=2, theta_0 = 0 and theta_1 = 1.

Some thoughts on re-design:

• Theta forecaster should not be a subclass of exponential smoothing
• We should isolate the components of this algorithm and then have theta forecaster just be a composition/pipeline made up of the components
  • Component 1. Deseasonalising and reseasonalising
  • Component 2. Idea of 'divide and conquer'. Creating a bunch of 'related time series' Zt from Yt, forecasting each of those Zt, and then combining these predictions of Zt to get prediction of Yt
  • Component 3. The particular 'related time series' Zt(theta) unique to theta forecasting. Note these are known as 'theta lines'

A big reason for this re-design is that AutoTheta will then just be a hyper-parameter optimisation. Explicitly:

• Let theta_0 = 0 and theta_1 is hyper-parameter to be optimised
• There is a hyper-parameter in component 1: additive or multiplicative deseasoning
• There is a hyper-parameter in component 2: whether the Zt should be combined additively or multiplicatively
• There is a hyper-parameter in component 3: whether Z(0) should be linear trend or exponential trend

Let me know thoughts or if something does not make sense. If something is vague, I recommend looking at the paper as it is well laid out and structured, and easy to read.

[fkiraly]

Hm, I need to have a look at this when I have more time. Also pinging @big-o who I believe originally implemented it.

Regarding the "isolation of components" idea, I very much like this - though it should be noted that there should also be a shorthand for the composite, and it may well be that it is more efficient to implement the composite as a whole, similar to time series forest etc.

[fkiraly]

ok, I've looked into this now in more detail.

• there are (at least) three relevant algorithms, "classic theta" (as referred to in the cited paper), generalized theta, and AutoTheta
• "classic theta" has n_theta=2, theta_1=0, theta_2=2 (not 1)
• the current implementation seems to implement, as you say, n_theta=2, theta_1=0, theta_2=1? And I'm not sure whether the mixture coefficients are right?

Regarding re-design, I agree we should extract the components, and aim at getting autotheta based on those components. This could replace PR #738.

I can see the following, interesting components:

• mixture of a forecast, with fixed weights/coefficients, WeightedAvgForecast. This is similar to the multiplexer, but with arbitrary weights.
• wrapping any supervised learner to fit y against time, and obtaining predictions by extrapolation. Linear regressor gives the theta-1 component. I.e., TrendForecaster, which is constructed with a supervised sklearn regressor inside
• differencing transformer, which applies differencing of d-th order before and inverts the differencing afterwards
• maybe a shorthand transformer ThetaLines or so

This way, we can obtain some interesting forecasters as contractions (assuming next refactor of pipeline):

• exponential forecast as ForecastingPipeline(LogTransform(), TrendForecaster(LinReg())
• generalized theta as using WeightedAvgForecast with specific components on multiple ThetaLines or other components
• autotheta by tuning generalized theta (e.g., using parameter tuning directly, or on a multiplexer/weighted average)

[Lovkush-A]

Alright! No small task. To structure this work, should we create one issue which has all these tasks listed in it, and then have multiple PRs relating to that one issue?

[fkiraly]

I suggest, let's prioritize - I think the TrendForecaster and WeightedAvgForecast might be most important, and the generalized theta - the rest may be easier to hand code first and/or can be added later.

Also, I'm not sure whether my sketch is the "best way" to go or whether it's missing something. @big-o or @mloning may have some comments.

[mloning]

There is some related functionality already:

• Deseasonalizer and Detrender for component 1
• EnsembleForecaster (takes multiple forecasters and not multiple series)
• PolynomialTrendForecaster (accepts any cross-sectional regression algorithm)
• LogTransformer
• @RNKuhns was thinking about the differencing transformer too and may have a draft already
• WeightedAvgForecast looks similar to the ColumnEnsemble we have for classification, where we train a separate algorithm on each variable and average predictions, reducing multivariate classification to univariate classification.

From this there are two main components missing:

• ThetaLines as a series-to-series transformer, taking a univariate series (pd.Series) as input and returning the theta lines (pd.DataFrame) if I understand this correctly
• ColumnEnsembleForecaster (or WeightedAvgForecast)

Does it make sense to have a ThetaForecaster class which implements the generalised theta model but defaults to the classic one? I'm not sure how hyper-parameter works in AutoTheta, whether it's based on a simple ForecastingGridSearchCV using predictive accuracy or whether it uses information criteria similar to AutoARIMA.

[mloning]

@Lovkush-A is this be something you have started implementing or could we assign this to one of the new contributors?

[Lovkush-A]

@mloning I have not started implementing this. Sure, if new contributor wants to do it, then please go ahead. I would be happy to review/help/collaborate with them if they wish.

[mloning]

We created an issue here #922