Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ENH] EnbPI based forecaster #6449

Merged
merged 23 commits into from
Jun 30, 2024
Merged

[ENH] EnbPI based forecaster #6449

merged 23 commits into from
Jun 30, 2024

Conversation

benHeid
Copy link
Contributor

@benHeid benHeid commented May 19, 2024

Reference Issues/PRs

Towards #6446, uses the aws-fortuna based approach together with tsbootstrap bootstraps

What does this implement/fix? Explain your changes.

Interfaces the aws-fortuna EnbPI implementation

Does your contribution introduce a new dependency? If yes, which one?

Yes aws-fortuna as well as third party library to use them (Jax==0.28.1)

What should a reviewer concentrate their feedback on?

Did you add any tests for the change?

get_test_params

Any other comments?

PR checklist

For all contributions
  • I've added myself to the list of contributors with any new badges I've earned :-)
    How to: add yourself to the all-contributors file in the sktime root directory (not the CONTRIBUTORS.md). Common badges: code - fixing a bug, or adding code logic. doc - writing or improving documentation or docstrings. bug - reporting or diagnosing a bug (get this plus code if you also fixed the bug in the PR).maintenance - CI, test framework, release.
    See here for full badge reference
  • Optionally, for added estimators: I've added myself and possibly to the maintainers tag - do this if you want to become the owner or maintainer of an estimator you added.
    See here for further details on the algorithm maintainer role.
  • The PR title starts with either [ENH], [MNT], [DOC], or [BUG]. [BUG] - bugfix, [MNT] - CI, test framework, [ENH] - adding or improving code, [DOC] - writing or improving documentation or docstrings.
For new estimators
  • I've added the estimator to the API reference - in docs/source/api_reference/taskname.rst, follow the pattern.
  • I've added one or more illustrative usage examples to the docstring, in a pydocstyle compliant Examples section.
  • If the estimator relies on a soft dependency, I've set the python_dependencies tag and ensured
    dependency isolation, see the estimator dependencies guide.

@benHeid benHeid added implementing algorithms Implementing algorithms, estimators, objects native to sktime module:forecasting forecasting module: forecasting, incl probabilistic and hierarchical forecasting labels May 19, 2024
@benHeid benHeid marked this pull request as ready for review June 9, 2024 07:30
@benHeid
Copy link
Contributor Author

benHeid commented Jun 9, 2024

@fkiraly should be ready for review :)

@benHeid
Copy link
Contributor Author

benHeid commented Jun 13, 2024

Before merging check the dependencies..

@benHeid
Copy link
Contributor Author

benHeid commented Jun 16, 2024

For the EnbPI wrapper, we using just a single method of the EnbPI class in the aws-fortuna package. Unfortunately this package has a lot of soft dependencies, including JAX. Furthermore, currently, the package is not easily installable due to dependency conflicts: awslabs/fortuna#194

Thus, I wonder if we could copy paste the method conformal_intervals method from the EnbPI class of the aws-fortuna package and adapt it slightly (adaption would get rid off the checks that aren't required anymore since it is internal and replacement of jax.numpy by numpy directly. If copying and modifiying we need to include the original license and the notice statement. Not sure how we should do this: https://github.com/awslabs/fortuna?tab=Apache-2.0-1-ov-file#readme and https://github.com/awslabs/fortuna/blob/main/NOTICE

@fkiraly any ideas?

@fkiraly
Copy link
Collaborator

fkiraly commented Jun 17, 2024

If it is a very small part only, it could go in a private submodule with a license file. We could also give the user the option to use fortuna if they manage to get it installed, via a hyperparameter, but that´s perhaps overkill.

Copy link
Collaborator

@fkiraly fkiraly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, some minor things:

  • for sktime estimators, the clone method (of the estimator) should be used, or configs will not be cloned.
  • the docstring should describe better how the algorithm is applied
  • I would move it to an enbpi module. compose is for pipelines or similar, it does have some "ensembles" in it but I think in some cases these are misnomers, in others I would not have put it there.

Non-blocking: some typos

@benHeid benHeid requested a review from fkiraly June 19, 2024 21:59
@benHeid
Copy link
Contributor Author

benHeid commented Jun 19, 2024

@fkiraly I addressed your changes and additionally also added a adapted version of the EnbPI implementation from aws-fortuna into the libs folder, together with the corresponding license and the NOTICE file that includes the copyright of aws-fortuna. I would appreciate a check if this is added correctly or if I should change the structure..

Copy link
Collaborator

@fkiraly fkiraly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regarding the library:

  • I would make it private, since it is not intended for public consumption
  • call it _aws_fortuna_enbpi or similar as it is not the entire library, just a method from it
  • README should be markdown, md
  • I would also suggest to add to the main README in libs. I would suggest to add a new section "snippets" or similar and explain that these are private
    • while you are at it, you could also fix the "distibution" type :-)

Also worth considering:

  • instead, new folder _libs_private, libs_private, or similar?
  • moving it into a folder close to the estimator, as it is the single dependency. Coupling-wise, might be better to use it directly from there?

What are your thoughts on handling private libraries?
Perhaps we should open an issue to discuss more widely with core devs, public and private libraries?

@benHeid
Copy link
Contributor Author

benHeid commented Jun 21, 2024

I addressed your comments and created a new issue #6648

fkiraly
fkiraly previously approved these changes Jun 21, 2024
@@ -5,6 +5,7 @@
__all__ = [
"HierarchyEnsembleForecaster",
"ColumnEnsembleForecaster",
"EnbPIForecaster",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

left over?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes..

@benHeid
Copy link
Contributor Author

benHeid commented Jun 23, 2024

@fkiraly Please wait before merging, I need to check if I introduced a Bug. I will ping you if everything is fine.

@benHeid
Copy link
Contributor Author

benHeid commented Jun 23, 2024

@fkiraly bug should be fixed now.

@fkiraly fkiraly merged commit 5c38c93 into main Jun 30, 2024
45 checks passed
@fkiraly fkiraly deleted the enbpi_based_forecaster branch June 30, 2024 15:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
implementing algorithms Implementing algorithms, estimators, objects native to sktime module:forecasting forecasting module: forecasting, incl probabilistic and hierarchical forecasting
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants