Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update extension template for predict_quantiles #1780

Merged
merged 3 commits into from Dec 22, 2021

Conversation

kejsitake
Copy link
Collaborator

What does this implement/fix? Explain your changes.

Updates the extension template to account for changes introducing predict_quantiles.

Does your contribution introduce a new dependency? If yes, which one?

No

Any other comments?

This was included in another PR #1579, but I am separating to make review easier. I have fixed some of the issues mentioned there.

@@ -90,8 +90,10 @@ class MyForecaster(BaseForecaster):
"requires-fh-in-fit": True, # is forecasting horizon already required in fit?
"X-y-must-have-same-index": True, # can estimator handle different X/y index?
"enforce_index_type": None, # index type that needs to be enforced in X/y
"capability:pred_int": False, # does forecaster implement predict_quantiles?
# in the future can be changed to capability:predict_quantiles
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bit unclear what this means to the person who wants to implement the forecaster. Perhaps say sth like "deprecated and will be renamed to capability:predict_quantiles in 0.11.0"?

Copy link
Collaborator

@fkiraly fkiraly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good!

Some comments:

  • docstring at the top should also be updated
  • one comment about deprecation

@kejsitake
Copy link
Collaborator Author

Hi! Thank you for the comments.

I made these changes and also removed the pred_int and alpha arguments from _update_predict_single and _predict_moving_cutoff. Please let me know if this isn't correct.

alpha : float or list, optional (default=0.95)
A significance level or list of significance levels.
fh : int, list, np.array or ForecastingHorizon
Forecasting horizon, default = y.index (in-sample forecast)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there is no y here, so the default does not seem to make sense - or does it and I'm missing sth?
If this is a typo, please remove the "default" part.

fkiraly
fkiraly previously approved these changes Dec 20, 2021
Copy link
Collaborator

@fkiraly fkiraly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

look good now.

I left one comment on the default fh, please fix if that's a typo.

@kejsitake
Copy link
Collaborator Author

Fixed the typo! Thank you for the comments!

Copy link
Collaborator

@fkiraly fkiraly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, looks great!

I have one more request, looking through this.
I don't think we should import from private modules here, i.e., the DEFAULT_ALPHA.

How I would avoid it is to set the default in the _predict_quantiles to None or, perhaps better, 0.5. In the body of the function, explain that alpha is guaranteed to be a float or list of float by predict_quantiles calling this function, which is by default the DEFAULT_ALPHA coming from predict_quantiles.

@kejsitake
Copy link
Collaborator Author

I believe I made this changes. I will keep in mind to include DEFAULT_ALPHA in the other PR refactoring predict_quantiles. Thank you!

Copy link
Collaborator

@fkiraly fkiraly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, thanks!

I noticed another thing, unfortunately, sorry for the bits-and-pieces here.

The docstring of _predict_quantiles is incorrect in describign the return, pred_quantiles. This is incorrectly referring to coverage, i.e., to the input of _predict_interval. Instead, it shoudl be referring to alpha.

@kejsitake
Copy link
Collaborator Author

kejsitake commented Dec 22, 2021

Thank you for the comments!
I changed the docstring and tried to read through the file for additional issues, could only spot some deprecated args in _update. But please let me know if there is anything else.

Copy link
Collaborator

@fkiraly fkiraly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, looks great now - thanks!

@fkiraly fkiraly merged commit 2a3ba75 into sktime:main Dec 22, 2021
@kejsitake kejsitake deleted the template-quantiles branch December 22, 2021 16:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants