Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ENH] ForecastingOptunaSearchCV for hyper-parameter tuning for forecasting #6630

Draft
wants to merge 30 commits into
base: main
Choose a base branch
from

Conversation

mk406
Copy link

@mk406 mk406 commented Jun 17, 2024

Reference Issues/PRs

Closes #6618. See also #4188

What does this implement/fix? Explain your changes.

While sktime provides a ForecastingGridSearchCV, ForecastingRandomizedSearchCV and ForecastingSkoptSearchCV for grid search, there's a need for more sophisticated and efficient hyper-parameter optimization methods. The lack of integration with advanced optimization libraries like Optuna limits the ability to perform more complex and efficient hyper-parameter searches.
We propose the addition of a ForecastingOptunaSearchCV class to sktime's forecasting module. This class would leverage Optuna for hyper-parameter optimization, providing a more efficient and flexible way to find the best parameters for forecasting models. The interface is similar to the existing ForecastingGridSearchCV, allowing users to easily switch between grid search and Optuna-based optimization without significant changes to their code.

Does your contribution introduce a new dependency? If yes, which one?

Adds a dependency on Optuna.

What should a reviewer concentrate their feedback on?

Did you add any tests for the change?

We added test_optuna in /sktime/sktime/forecasting/model_selection/tests/test_tune.py

Any other comments?

PR checklist

For all contributions
  • I've added myself to the list of contributors with any new badges I've earned :-)
    How to: add yourself to the all-contributors file in the sktime root directory (not the CONTRIBUTORS.md). Common badges: code - fixing a bug, or adding code logic. doc - writing or improving documentation or docstrings. bug - reporting or diagnosing a bug (get this plus code if you also fixed the bug in the PR).maintenance - CI, test framework, release.
    See here for full badge reference
  • Optionally, for added estimators: I've added myself and possibly to the maintainers tag - do this if you want to become the owner or maintainer of an estimator you added.
    See here for further details on the algorithm maintainer role.
  • The PR title starts with either [ENH], [MNT], [DOC], or [BUG]. [BUG] - bugfix, [MNT] - CI, test framework, [ENH] - adding or improving code, [DOC] - writing or improving documentation or docstrings.
For new estimators
  • I've added the estimator to the API reference - in docs/source/api_reference/taskname.rst, follow the pattern.
  • I've added one or more illustrative usage examples to the docstring, in a pydocstyle compliant Examples section.
  • If the estimator relies on a soft dependency, I've set the python_dependencies tag and ensured
    dependency isolation, see the estimator dependencies guide.

@fkiraly fkiraly added implementing algorithms Implementing algorithms, estimators, objects native to sktime module:forecasting forecasting module: forecasting, incl probabilistic and hierarchical forecasting enhancement Adding new functionality labels Jun 17, 2024
@fkiraly
Copy link
Collaborator

fkiraly commented Jun 18, 2024

Code formatting tests are failing, see here how to automate code formatting for sktime code: https://www.sktime.net/en/stable/developer_guide/coding_standards.html

@fkiraly fkiraly changed the title Adding ForecastingOptunaSearchCV for hyper-parameter tuning for forecasting [ENH] ForecastingOptunaSearchCV for hyper-parameter tuning for forecasting Jun 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Adding new functionality implementing algorithms Implementing algorithms, estimators, objects native to sktime module:forecasting forecasting module: forecasting, incl probabilistic and hierarchical forecasting
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[ENH] Adding ForecastingOptunaSearchCV for hyper-parameter tuning for forecasting
3 participants