Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ENH] Forecasters: prediction interval, probabilistic forecasting interface refactor #1687

Closed
27 tasks done
fkiraly opened this issue Dec 3, 2021 · 4 comments
Closed
27 tasks done

[ENH] Forecasters: prediction interval, probabilistic forecasting interface refactor #1687

fkiraly opened this issue Dec 3, 2021 · 4 comments
Assignees
@fkiraly @kejsitake
Labels
module:forecasting forecasting module: forecasting, incl probabilistic and hierarchical forecasting
Projects
Workstream: forecasting and series tr...

Comments

@fkiraly
Copy link
Collaborator

fkiraly commented Dec 3, 2021 •
edited

We're currently working on a major refactor of the forecasting core interfaces to allow probabilistic and interval predictions.
The design is here:
https://github.com/sktime/enhancement-proposals/blob/main/steps/13_proba_forecasting/step.md

Major work items:

Design:

Interface/framework refactor:

  • framework: predict_interval and predict_quantiles
  • framework: predict_var
  • framework: predict_proba

Refactoring or adding individual methods:

Capstone activities:

  • refactor capstone
    • predict_interval and predict_quantiles
    • predict_var
    • predict_proba
  • deprecation complete
@fkiraly fkiraly added the module:forecasting forecasting module: forecasting, incl probabilistic and hierarchical forecasting label Dec 3, 2021
@kejsitake
Copy link
Collaborator

kejsitake commented Dec 9, 2021 •
edited by fkiraly

According to my initial checks the forecastors that should have probabilistic capabilities and still need to be refactored are:

@kejsitake kejsitake mentioned this issue Dec 12, 2021
Closed
@fkiraly fkiraly moved this from In progress to Review in progress in Workstream: forecasting and series transformers Dec 17, 2021
@IlyasMoutawwakil IlyasMoutawwakil mentioned this issue Jan 12, 2022
Merged
6 tasks
@kejsitake kejsitake mentioned this issue Jan 14, 2022
Merged
3 tasks
@fkiraly
Copy link
Collaborator Author

fkiraly commented Jan 18, 2022

@kejsitake, I just tried to prepare the refactor capstone (for deprecation).
It would look like we forgot Prophet!

@fkiraly fkiraly mentioned this issue Jan 21, 2022
Merged
@fkiraly
Copy link
Collaborator Author

fkiraly commented Jan 21, 2022

From our discussion on slack: @kejsitake has tentatively taken Prophet on for refactoring.

@fkiraly fkiraly moved this from Review in progress to In progress in Workstream: forecasting and series transformers Jan 21, 2022
@kejsitake kejsitake mentioned this issue Jan 26, 2022
Merged
@fkiraly fkiraly mentioned this issue Feb 20, 2022
Merged
fkiraly added a commit that referenced this issue Mar 20, 2022
@fkiraly
[ENH] predict_proba for forecasters, tensorflow-probability depen…
061b170 
…dency (#2100)

Relates to #1746, proposes interface and technical solution.
Solves interface/framework for `predict_var` and `predict_proba` in #1687.

This PR adds `predict_proba` and `predict_var` interface and default behaviour to `BaseForecaster`:
* `predict_var` returns a `pd.DataFrame` of variance predictions
* `predict_proba` returns a `tfp` `Distribution` with batch indices corresponding to `fh` indices

The default behaviour is as follows:
* `predict_var` first looks if `predict_proba` is implemented, and gets variance there if it is. Otherwise, if intervals/quantiles are implemented, returns variance of a normal with same IQR as the predictions
* `predict_proba` uses `predict` as location and square root of `predict_var` as scale parameter. If `predict_var` is not directly implemented, leverages its default behaviour to get this from intervals/quantiles.

This PR also introduces a soft dependency on `tensorflow` and `tensorflow-probability`, localized to a `dl` dependency set.
This dependency set is added to dependency specifications where necessary; a function `_check_dl_dependencies` to check and raise error if the dependency is not installed is introduced.

Further, this PR turns on a number of tests to work for `predict_var` and `predict_proba`, by adding defaults for these methods to the scenarios.

Depends on the following PR:
* #2105
* #2108
@fkiraly
Copy link
Collaborator Author

fkiraly commented Mar 25, 2022

All done:

@fkiraly fkiraly closed this as completed Mar 25, 2022
Workstream: forecasting and series transformers automation moved this from In progress to Done Mar 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@fkiraly fkiraly

@kejsitake kejsitake

Labels
module:forecasting forecasting module: forecasting, incl probabilistic and hierarchical forecasting
Projects
Workstream: forecasting and series tr...
Done
Milestone
No milestone
Development

No branches or pull requests
3 participants
@fkiraly @kejsitake @SveaMeyer13