Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ENH] MultiplexForecaster compatibility with multivariate, probabilistic and hierarchical forecasting #2458

Merged
merged 26 commits into from Apr 28, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
13448fe
multiplexer using delegate
fkiraly Apr 13, 2022
4153cb6
improved docstring
fkiraly Apr 13, 2022
f726e79
use heterogeneousensemble methods
fkiraly Apr 13, 2022
8f04e33
docstring
fkiraly Apr 13, 2022
e016493
fixed clone ref, added None param
fkiraly Apr 13, 2022
44b77c5
removing EnsembleForecaster parent
fkiraly Apr 13, 2022
93911ac
multiplexer fit is not empty
fkiraly Apr 13, 2022
1f3be57
refactor update_predict
fkiraly Apr 14, 2022
45c4125
Merge branch 'update_predict_first_refactor' into multiplexer-proba
fkiraly Apr 14, 2022
4d38642
Merge branch 'main' into multiplexer-proba
fkiraly Apr 14, 2022
d8ee716
Merge branch 'main' into multiplexer-proba
fkiraly Apr 21, 2022
849ac5c
fixed bug found by miraep8 test suite
fkiraly Apr 22, 2022
0db3b71
Merge branch 'main' into multiplexer-proba
fkiraly Apr 22, 2022
30b4fbf
added missing docstring for forecasters_
fkiraly Apr 24, 2022
60dba50
Merge branch 'main' into multiplexer-proba
fkiraly Apr 24, 2022
fd91d0d
improve docstring clarity
fkiraly Apr 25, 2022
aff2358
om
fkiraly Apr 25, 2022
7688352
deep default
fkiraly Apr 25, 2022
802dd15
comment on _DelegatedForecaster attribute
fkiraly Apr 25, 2022
a024bbf
Merge branch 'main' into multiplexer-proba
fkiraly Apr 25, 2022
32a8619
typo in get/set params
fkiraly Apr 25, 2022
faf6e4a
test with only list
fkiraly Apr 25, 2022
0f2b2aa
Merge branch 'main' into multiplexer-proba
fkiraly Apr 26, 2022
41ec855
Merge branch 'main' into multiplexer-proba
fkiraly Apr 27, 2022
1d09e70
removed _fit since reset is now done in fit
fkiraly Apr 27, 2022
57ca452
Merge branch 'main' into multiplexer-proba
fkiraly Apr 27, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
3 changes: 2 additions & 1 deletion sktime/base/_meta.py
Expand Up @@ -268,7 +268,8 @@ def _get_estimator_tuples(self, estimators, clone_ests=False):
Parameters
----------
estimators : list of estimators, or list of (str, estimator tuples)
clone_ests : bool, whether estimators get cloned in the process
clone_ests : bool, optional, default=False.
whether estimators of the return are cloned (True) or references (False)

Returns
-------
Expand Down
2 changes: 1 addition & 1 deletion sktime/classification/compose/_pipeline.py
Expand Up @@ -274,7 +274,7 @@ def get_params(self, deep=True):

Parameters
----------
deep : boolean, optional
deep : boolean, optional, default=True
If True, will return the parameters for this estimator and
contained sub-objects that are estimators.

Expand Down
2 changes: 1 addition & 1 deletion sktime/forecasting/base/_meta.py
Expand Up @@ -94,7 +94,7 @@ def get_params(self, deep=True):

Parameters
----------
deep : boolean, optional
deep : boolean, optional, default=True
If True, will return the parameters for this estimator and
contained sub-objects that are estimators.

Expand Down
155 changes: 81 additions & 74 deletions sktime/forecasting/compose/_multiplexer.py
Expand Up @@ -5,13 +5,15 @@

from sklearn.base import clone

from sktime.forecasting.base._meta import _HeterogenousEnsembleForecaster
from sktime.base import _HeterogenousMetaEstimator
from sktime.forecasting.base._base import BaseForecaster
from sktime.forecasting.base._delegate import _DelegatedForecaster

__author__ = ["kkoralturk", "aiwalter"]
__author__ = ["kkoralturk", "aiwalter", "fkiraly"]
__all__ = ["MultiplexForecaster"]


class MultiplexForecaster(_HeterogenousEnsembleForecaster):
class MultiplexForecaster(_DelegatedForecaster, _HeterogenousMetaEstimator):
"""MultiplexForecaster for selecting among different models.

MultiplexForecaster facilitates a framework for performing
Expand All @@ -20,38 +22,39 @@ class MultiplexForecaster(_HeterogenousEnsembleForecaster):
to get full utilization. It can be used with univariate and
multivariate forecasters.

Single use of MultiplexForecaster with forecasters
and selected_forecaster parameter specified,
works just like the selected component.
It does not provide any further use in that case.
MultiplexForecaster is specified with a (named) list of forecasters
and a selected_forecaster hyper-parameter, which is one of the forecaster names.
The MultiplexForecaster then behaves precisely as the forecaster with
name selected_forecaster, ignoring functionality in the other forecasters.

When used with ForecastingGridSearchCV, MultiplexForecaster
provides an ability to compare different model class
performances with each other, just like a model tournament.
When ForecastingGridSearchCV is fitted with a MultiplexForecaster,
returned value for the selected_forecaster argument of best_params_
attribute of ForecastingGridSearchCV, gives the best
performing model class among given models provided in forecasters.
provides an ability to tune across multiple estimators, i.e., to perform AutoML,
by tuning the selected_forecaster hyper-parameter. This combination will then
select one of the passed forecasters via the tuning algorithm.

Parameters
----------
forecasters : list
List of (forecaster names, forecaster objects)
MultiplexForecaster switches between these forecasters
objects when used with ForecastingGridSearchCV to
find the optimal model
selected_forecaster: str
An argument to make a selection among forecasters.
MultiplexForecaster uses selected_forecaster
to choose which component to fit.
Important for using with ForecastingGridSearchCV as a
hyperparameter.
forecasters : list of sktime forecasters, or
list of tuples (str, estimator) of sktime forecasters
MultiplexForecaster can switch ("multiplex") between these forecasters.
These are "blueprint" forecasters, states do not change when `fit` is called.
selected_forecaster: str or None, optional, Default=None.
If str, must be one of the forecaster names.
If no names are provided, must coincide with auto-generated name strings.
To inspect auto-generated name strings, call get_params.
If None, behaves as if the first forecaster in the list is selected.
Selects the forecaster as which MultiplexForecaster behaves.

Attributes
----------
forecaster_ : Sktime forecaster
forecaster that MultiplexForecaster will currently
forecast with.
forecaster_ : sktime forecaster
clone of the selected forecaster used for fitting and forecasting.
forecasters_ : list of (str, forecaster) tuples
str are identical to those passed, if passed strings are unique
otherwise unique strings are generated from class name; if not unique,
the string `_[i]` is appended where `[i]` is count of occurrence up until then
forecasters in `forecasters_`are reference to forecasters in arg `forecasters`
i-th forecaster in `forecasters_` is clone of i-th in `forecasters`

Examples
--------
Expand Down Expand Up @@ -84,83 +87,80 @@ class MultiplexForecaster(_HeterogenousEnsembleForecaster):
"handles-missing-data": False,
"scitype:y": "both",
"y_inner_mtype": ["pd.DataFrame", "pd.Series"],
"fit_is_empty": False,
}

# attribute for _DelegatedForecaster, which then delegates
# all non-overridden methods to those of same name in self.forecaster_
# see further details in _DelegatedForecaster docstring
_delegate_name = "forecaster_"
fkiraly marked this conversation as resolved.
Show resolved Hide resolved

def __init__(
self,
forecasters: list,
selected_forecaster=None,
):
super(MultiplexForecaster, self).__init__(forecasters=forecasters, n_jobs=None)
super(MultiplexForecaster, self).__init__()
self.selected_forecaster = selected_forecaster

self.forecasters = forecasters
fkiraly marked this conversation as resolved.
Show resolved Hide resolved
self.forecasters_ = self._check_estimators(
forecasters,
attr_name="forecasters",
cls_type=BaseForecaster,
clone_ests=False,
)
self._set_forecaster()
fkiraly marked this conversation as resolved.
Show resolved Hide resolved
self.clone_tags(self.forecaster_)
self.set_tags(**{"fit_is_empty": False})

fkiraly marked this conversation as resolved.
Show resolved Hide resolved
def _check_selected_forecaster(self):
component_names = [name for name, _ in self.forecasters]
if self.selected_forecaster not in component_names:
component_names = self._get_estimator_names(self.forecasters_, make_unique=True)
selected = self.selected_forecaster
if selected is not None and selected not in component_names:
raise Exception(
"Please check the selected_forecaster argument provided "
" Valid selected_forecaster parameters: {}".format(component_names)
f"Invalid selected_forecaster parameter value provided, "
f" found: {self.selected_forecaster}. Must be one of these"
f" valid selected_forecaster parameter values: {component_names}."
)

def _set_forecaster(self):
self._check_selected_forecaster()
# clone the selected forecaster to self.forecaster_
if self.selected_forecaster is not None:
for name, forecaster in self.forecasters:
for name, forecaster in self._get_estimator_tuples(self.forecasters):
if self.selected_forecaster == name:
self.forecaster_ = clone(forecaster)
else:
# if None, simply clone the first forecaster to self.forecaster_
fkiraly marked this conversation as resolved.
Show resolved Hide resolved
self.forecaster_ = clone(self._get_estimator_list(self.forecasters)[0])

def _fit(self, y, X=None, fh=None):
"""Fit to training data.

Parameters
----------
y : pd.Series, pd.DataFrame
Target time series to which to fit the forecaster.
fh : int, list or np.array, optional (default=None)
The forecasters horizon with the steps ahead to to predict.
X : pd.DataFrame, optional (default=None)
Exogenous variables are ignored

Returns
-------
self : returns an instance of self.
"""
self._check_forecasters()
self._set_forecaster()
self.forecaster_.fit(y, X=X, fh=fh)
return self

def _predict(self, fh, X=None):
"""Forecast time series at future horizon.
def get_params(self, deep=True):
"""Get parameters for this estimator.

Parameters
----------
fh : int, list, np.array or ForecastingHorizon
Forecasting horizon
X : pd.DataFrame, optional (default=None)
Exogenous time series
deep : boolean, optional, default=True
If True, will return the parameters for this estimator and
contained subobjects that are estimators.

Returns
-------
y_pred : pd.Series
Point predictions
params : mapping of string to any
Parameter names mapped to their values.
"""
return self.forecaster_.predict(fh=fh, X=X)
return self._get_params("forecasters_", deep=deep)

def _update(self, y, X=None, update_params=True):
"""Call predict on the forecaster with the best found parameters.
def set_params(self, **kwargs):
"""Set the parameters of this estimator.

Parameters
----------
y : pd.Series, pd.DataFrame
X : pd.DataFrame, optional (default=None)
update_params : bool, optional (default=True)
Valid parameter keys can be listed with ``get_params()``.

Returns
-------
self : an instance of self
self
"""
self.forecaster_.update(y, X, update_params=update_params)
self._set_params("forecasters_", **kwargs)
return self

@classmethod
Expand All @@ -179,12 +179,19 @@ def get_test_params(cls, parameter_set="default"):
"""
from sktime.forecasting.naive import NaiveForecaster

params = {
params1 = {
"forecasters": [
("Naive_mean", NaiveForecaster(strategy="mean")),
("Naive_last", NaiveForecaster(strategy="last")),
("Naive_drift", NaiveForecaster(strategy="drift")),
],
"selected_forecaster": "Naive_mean",
}
return params
params2 = {
"forecasters": [
NaiveForecaster(strategy="mean"),
NaiveForecaster(strategy="last"),
NaiveForecaster(strategy="drift"),
],
}
return [params1, params2]
2 changes: 1 addition & 1 deletion sktime/forecasting/compose/_pipeline.py
Expand Up @@ -162,7 +162,7 @@ def get_params(self, deep=True):

Parameters
----------
deep : boolean, optional
deep : boolean, optional, default=True
If True, will return the parameters for this estimator and
contained subobjects that are estimators.

Expand Down
4 changes: 2 additions & 2 deletions sktime/transformations/compose.py
Expand Up @@ -346,7 +346,7 @@ def get_params(self, deep=True):

Parameters
----------
deep : boolean, optional
deep : boolean, optional, default=True
If True, will return the parameters for this estimator and
contained sub-objects that are estimators.

Expand Down Expand Up @@ -623,7 +623,7 @@ def get_params(self, deep=True):

Parameters
----------
deep : boolean, optional
deep : boolean, optional, default=True
If True, will return the parameters for this estimator and
contained sub-objects that are estimators.

Expand Down