Allow constant trend

[luofanghao]

Hi,

Is there a way to force the trend to be flat? My data has strong daily and weekly seasonality, rather than a strong trend change pattern. Is there a way to make a flat trend on model?

example:

right now the trend change is too strong, so if I use the current model to predict, prediction will go down (because of the trend). I want to avoid this by making the trend flat. What really matters in my data should only be the seasonalities.

[AvielMak]

Have you looked at changepoint_prior_scale this should help you adjust the trend flexibility
https://facebook.github.io/prophet/docs/trend_changepoints.html

[luofanghao]

@mcaviel thank you.

I have tried but turn down changepoint_prior_scale does not really help. And the acceptable smallest value for n_changepoint is 1. So the trend still not be flat.

[luofanghao]

I am thinking about taking the mean value of predicted trend values, and manually re-calculated the yhat value. This might work. But I haven't figure out the formula yet. Does anyone know it?

I tried this one but it is not:

fcst["yhat_upper"] - fcst["trend_upper"]*fcst["multiplicative_terms_upper"]*( fcst["additive_terms_upper"]+fcst["daily_upper"] + fcst["weekly_upper"])

[bletham]

This is a challenging time series for Prophet since it does not conform very well to the model.

The multiplicative Prophet model is that y = trend * (daily_seasonality + weekly_seasonality). The model assumes that the seasonalities are constant and independent of eachother. That is, the daily seasonality (seasonality within one day) is the same every day. That is clearly not the case here - the daily seasonality is stronger on some days than on others. Having interactions between seasonalities is in #538, and there is a solution there but it's a bit of effort.

Daily seasonality that depends on weekly seasonality is one challenge with this time series. The other challenge is that the middle week has stronger effects than the other two weeks. Since the daily and weekly seasonalities are constant, the only way for the model to have higher values on the middle week is via the trend.

You could try to resolve the issue with the trend decreasing in the future by using logistic growth and setting a floor (see the bottom of https://facebook.github.io/prophet/docs/saturating_forecasts.html ). But I think you'd have to follow the instructions in #538 to resolve the seasonalities interaction before you could get a reasonable forecast out of Prophet for this.

[bletham]

Oh, and I forgot to mention - there is currently no way to specify a constant trend. I think there is an issue open for this but can't find it at the moment.

[luofanghao]

@bletham thank you so much for the concrete information.

And you are right, I realized that my data does not conform very well to the model, especially if I want to make a constant trend. However, it is okay that the "constant-trend model" does not fit the data very well. Because what I am doing is an anomaly detection project, and actually the middle week should be considered to be anomaly. That is what I mean, even seems like there is a trend, but it actually is not.

[bletham]

That's reasonable, and a flat trend seems like a good thing to support in the future. In the meantime, the only way this could be done would be by modifying the Stan code to put a very tight prior on k that forces it to be close to zero. (Currently there is no prior on k).

[luofanghao]

@bletham Thank you. Hope we can have this new feature in the future.

[kaleming]

Now with #501, is there a easy way (changing k) to make this possible ?

Could you give some hint how to do that ?

[sammo]

I believe Ben means we need to set the prior scale for k to something much smaller than 5, what it currently is set to here

prophet/python/stan/unix/prophet.stan

Line 109 in a3ae471

k ~ normal(0, 5);

Please correct me if I'm wrong @bletham .
I tried doing that and it sure does help set a flat trend. I'm facing a few questions though. It seems like it it works better for a logistic growth? It also seems like a very low scale for the trend would throw the optimizer in a limbo more often. That was the case for scale values 0.001 and lower. Higher scales are resulting in a significantly positive to negative growth coefficient... Any insight on what should this number be to get a flat linear trend, @bletham ?

[bletham]

@sammo Yes, k is the initial slope and so a very tight prior will force it to be very close to 0. And then if n_changepoints=0, it will stay flat at 0. There is another parameter m that sets the y intercept and that will still be fit.

I'd expect that to work well for the linear trend. For the logistic I have less intuition - there isn't a y-intercept term in the logistic growth function rather there is an x offset, so I think taking k close to 0 would force the trend to be 0.5 * cap? If this is a bad fit then that would definitely mess up the optimizer.

If you're anyway modifying Stan code, then there's a cleaner way of doing this by just directly editing the trend function here:

prophet/python/stan/unix/prophet.stan

Line 74 in a3ae471

return (k + A * delta) .* t + (m + A * (-t_change .* delta));

If you replace that with

return m;

then it will fit a flat trend with y-intercept m (which is fit). The same modification has to be made in the Py/R definition of the trend, which is used at predict time:

prophet/python/fbprophet/forecaster.py

Line 1199 in a3ae471

def piecewise_linear(t, deltas, k, m, changepoint_ts):

That entire function would be replaced by

return m * np.ones_like(t)

to match the modified Stan.

That requries 2 edits but doesn't make you have to worry about super tight priors or inference issues that may come as a result.

[sammo]

@bletham thanks for the thorough explanation. That makes sense. There might be a few cases where this would be helpful for some of the time series I'm working with so I may use it...

[MrBordeaux]

I am not sure how elegant it is, but you can get prophet to force a constant trend by setting growth='logistic', cap = last * (1 + epsilon), floor = last * (1 - epsilon), where last is the latest y-value you have, and epsilon is a very small number

[Rosina9700]

Just adding a +1 that being able to manually configure the growth curve would be great. I'm also attempting to use Prophet for anomaly detection and find that often, the anomalous behavior leads to unwanted growth trends. I appreciate that this is great functionality for forecasting but not so great for anomaly detection (which is not the intended case for Prophet). But if I could fix the trend to be zero, then the anomalous period would be easier to identify and Prophet would very applicable to anomaly detection too.

[ryankarlos]

@Rosina9700 Ive just submitted PR #1466 for this - although my working solution sets a flat growth rather than allowing tightening of the growth prior, but seems like thats what people are keen on anyway.

[Rosina9700]

@ryankarlos and @bletham thanks for tackling this and getting it merged! Looking forwards to trying it out!

[bletham]

@Rosina9700 if you do try this out and have a nice example that you can share the data for, please do post it so we could potentially use it in the documentation!

[bletham]

I'm leaving this open until it gets ported to R too.

[bletham]

This feature is now in R and in CRAN in the latest v1.0 release.