Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Theta forecaster and prediction intervals #198

Merged
merged 37 commits into from
Feb 3, 2020

Conversation

big-o
Copy link
Contributor

@big-o big-o commented Nov 17, 2019

Reference Issues/PRs

#67 #97

What does this implement/fix? Explain your changes.

Includes:

  • ThetaForecaster which implements the Theta method.
  • Unit tests for ThetaForecaster.
  • A new dataset in datasets.load_airline - the Box/Jenkins airline passengers dataset. I thought this provided a clearer example of simple, univariate, seasonal data than some of the other datasets currently in sktime.
  • New interface in BaseForecaster to enable prediction intervals to be calculated: prediction_errors. Currently this has only been implemented for ThetaForecaster. Running this method for any other forecaster will raise a NotImplementedError.
  • Alternatively, prediction intervals may also be obtained by calling predict() and providing one or many confidence levels. This will cause the method to return a tuple of predictions and errors, instead of just predictions.
  • A plot() convenience method for forecasters. As well as plotting the forecasts (optionally alongside any training/test data), it will also plot prediction intervals if available for that forecaster.
  • A new notebook, simple_forecasting.ipynb, to demonstrate all of the above.

Any other comments?

Other projects I looked at like as pmdarima and R's forecast library return upper and lower bounds for prediction intervals. I've opted for a different approach though, of just doing the bare minimum and calculating the forecast point errors and returning this as a single series. It's then trivial to apply this to your forecasts to obtain the intervals. This is what the new plot() method does under the hood. I think it should still be fairly easy to interface with some of these other packages even with this different approach but please let me know if you disagree.

FYI, prediction_errors supports the calculation of producing PIs at different confidence levels, something that was discussed in #97. It also allows each forecaster to define its own methods for PI calculations. I didn't attempt to implement this for any of the other forecasters as this could be quite a lot of code for some of them (in the case of ETS forecasting, I believe some models don't even have a known formula - for this reason I implemented the interface in a way that makes it optional).

I thought about merging in the new notebook with the existing forecasting notebook, but wasn't sure how to best make them fit together. I'm happy to take a steer on this if you'd prefer me to put all the forecasting stuff in a single notebook before merging the PR.

PR checklist

  • I have added unit tests and made sure they pass locally.
  • I have updated the existing example notebooks or provided a new one to showcase how my algorithm works.
  • I have updated sktime's estimator overview and I'm committed to maintain my contribution and provide bug fixes if necessary.

@big-o big-o changed the title Theta forecaster Theta forecaster and prediction intervals Nov 17, 2019
@mloning mloning added this to Review in progress in Workstream: forecasting and series transformers via automation Nov 18, 2019
@mloning mloning added the feature request New feature or request label Nov 18, 2019
@mloning mloning self-assigned this Nov 18, 2019
@mloning mloning changed the base branch from forecasting to fcast January 30, 2020 08:46
Copy link
Contributor

@mloning mloning left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work! I've added a few minor comments!

sktime/forecasting/base.py Outdated Show resolved Hide resolved
sktime/forecasting/base.py Show resolved Hide resolved
sktime/forecasting/base.py Outdated Show resolved Hide resolved
sktime/forecasting/base.py Outdated Show resolved Hide resolved
sktime/forecasting/base.py Outdated Show resolved Hide resolved
sktime/forecasting/exp_smoothing.py Outdated Show resolved Hide resolved
sktime/forecasting/exp_smoothing.py Outdated Show resolved Hide resolved
sktime/forecasting/theta.py Outdated Show resolved Hide resolved
@big-o big-o mentioned this pull request Jan 30, 2020
2 tasks
@mloning mloning added implementing algorithms Implementing algorithms, estimators, objects native to sktime and removed feature request New feature or request labels Jan 31, 2020
@mloning
Copy link
Contributor

mloning commented Feb 3, 2020

Nice, great work!

@mloning mloning merged commit f3efcb2 into sktime:fcast Feb 3, 2020
Workstream: forecasting and series transformers automation moved this from Review in progress to Done Feb 3, 2020
@big-o big-o deleted the theta-forecaster branch February 3, 2020 12:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
implementing algorithms Implementing algorithms, estimators, objects native to sktime
Development

Successfully merging this pull request may close these issues.

None yet

2 participants