[ENH] Adds _predict_interval to SARIMAX to support predict_interval and predict_quantiles

[yarnabrina]

Reference Issues/PRs

Fixes #4301.

What does this implement/fix? Explain your changes.

Before this PR, SARIMAX did not support prediction of confidence intervals and quantiles. This PR adds those functionalities, and tests to check that results match with underlying statsmodels estimator.

Does your contribution introduce a new dependency? If yes, which one?

No.

What should a reviewer concentrate their feedback on?

Did you add any tests for the change?

Yes.

Any other comments?

• SARIMAX in statsmodels has a boolean argument simple_differencing. If it's set to be True, the forecasts and predictions are about the differenced series. It can be considered to add handling of that, similar to how pmdarima does it.
• SARIMAX did return datetime index for me for the example in [BUG] DynamicFactor returns incorrect index for probabilistic forecasts #4362, and the available methods seem different as well. So, skipped generalisation attempt as of now as suggested here.

PR checklist

For all contributions

• I've added myself to the list of contributors.
• Optionally, I've updated sktime's CODEOWNERS to receive notifications about future changes to these files.
• The PR title starts with either [ENH], [MNT], [DOC], or [BUG] indicating whether the PR topic is related to enhancement, maintenance, documentation, or bug.

For new estimators

• I've added the estimator to the online documentation.
• I've updated the existing example notebooks or provided a new one to showcase how my estimator works.

[fkiraly]

Great, as usual!

Non-blocking comments:

• you should add yourself as an author
• the _predict_interval looks very similar to that in other statsmodels functions. It might make sense to see whether the duplications (or partial duplications) can be refactored to live in a single place. I would consider that out of scope though for this PR, just making a note.

[fkiraly]

Regarding questions:

It can be considered to add handling of that

Yes, that would make sense.
If you look at that, it would be great if you also could add the missing fit args, see here: #4377

skipped generalisation attempt as of now as suggested #4301 (comment).

I suppose that's the same comment I just made in the review? Fair enough, so what you're saying, there are subtle interface inconsistencies?

[fkiraly]

Another thing I noticed, apologies for not letting you know first time round - I just fixed something similar as part of the pandas 2 compatibility patch.

lines 242, 243 look identical to lines in AutoETS.
These cause a failure on pandas 2 which no longer allows to write to index locations that do not exist.
Could you kindly follow the fix in #4427 (AutoETS) instead? (this replaces the offending lines by an alternative call that works on both pandas 1 and pandas 2)

I also notice, the entire _predict_interval looks identical, except the name of the method of prediction_results which is called.
Could a refactor work by adding an attribute or tag to the adapter parent class which has the name of the attribute (string) and is looked up inside _predict_interval? Or a private function with that as an argument, which is called from _predict_interval?
Would appreciate an issue with any observations you might have on similarities and/or how to refactor this (if you don't get to it, I will open one later when I'm done with the pandas 2 upgrade)

[yarnabrina]

what you're saying, there are subtle interface inconsistencies?

Yes. For example, DynamicFactor has a method predict_interval that can be used, while SARIMAX does not and one has to use get_prediction or similar first and on top that use conf_int. I can't think of a way to merge these.

[fkiraly]

I can't think of a way to merge these.

Well, but there are at least three instances where the similarities are very high, see above.
We can deal with these, and override _predict_interval in cases of substantial difference, e.g., DynamicFactor.

(maybe we should move this discussion to a refactor issue, as it isn't really about #4439 anymore)

[fkiraly]

Thanks, that's exactly what I meant!