Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ENH] tsbootstrap adapter #5887

Merged
merged 36 commits into from Mar 22, 2024
Merged

[ENH] tsbootstrap adapter #5887

merged 36 commits into from Mar 22, 2024

Conversation

benHeid
Copy link
Contributor

@benHeid benHeid commented Feb 3, 2024

Reference Issues/PRs

Do not merge before release related issues are fixed in astrogilda/tsbootstrap

closes #5880
closes astrogilda/tsbootstrap#45

What does this implement/fix? Explain your changes.

Add an adapter for tsbootstrap

Does your contribution introduce a new dependency? If yes, which one?

tsbootstrap

What should a reviewer concentrate their feedback on?

Did you add any tests for the change?

Any other comments?

PR checklist

For all contributions
  • I've added myself to the list of contributors with any new badges I've earned :-)
    How to: add yourself to the all-contributors file in the sktime root directory (not the CONTRIBUTORS.md). Common badges: code - fixing a bug, or adding code logic. doc - writing or improving documentation or docstrings. bug - reporting or diagnosing a bug (get this plus code if you also fixed the bug in the PR).maintenance - CI, test framework, release.
    See here for full badge reference
  • The PR title starts with either [ENH], [MNT], [DOC], or [BUG]. [BUG] - bugfix, [MNT] - CI, test framework, [ENH] - adding or improving code, [DOC] - writing or improving documentation or docstrings.
For new estimators
  • I've added the estimator to the API reference - in docs/source/api_reference/taskname.rst, follow the pattern.
  • I've added one or more illustrative usage examples to the docstring, in a pydocstyle compliant Examples section.
  • If the estimator relies on a soft dependency, I've set the python_dependencies tag and ensured
    dependency isolation, see the estimator dependencies guide.

@benHeid benHeid added module:transformations transformations module: time series transformation, feature extraction, pre-/post-processing enhancement Adding new functionality labels Feb 3, 2024

Parameters
----------
tsbootstrapper : Bootstrapper from tsbootstrap
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

too long variable name imo

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is bootstrapper short enough?

Copy link
Collaborator

@fkiraly fkiraly Feb 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe bootstrap?
(that's the shorthand and scitype name I've used in the refactor,, and less strange in English than "bootstrapper")

"enforce_index_type": None, # index type that needs to be enforced in X/y
"fit_is_empty": True, # is fit empty and can be skipped? Yes = True
"transform-returns-same-time-index": False,
"python_dependencies": "tsbootstrap",
Copy link
Collaborator

@fkiraly fkiraly Feb 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

>=0.1.0?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean something like: "tsbootstrap>=0.1.0"? I tested it with "tsbootstrap>=0.0.32 and this is failing...

Not sure, but I think the dependency check is not correct here: https://github.com/sktime/sktime/blob/63a95f2c29251575b7963ad269044654cbe9d96c/sktime/utils/validation/_dependencies.py#L177C1-L178C1

bootstrapped_samples, index=multi_index, columns=X.columns
)

X.index = pd.MultiIndex.from_product([["original"], X.index])
Copy link
Collaborator

@fkiraly fkiraly Feb 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we always want to include the original series?

I think it should be at most optional, parameter controlled, default=no

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mhm.. I used the behavior of the STLBootstrap as template. This bootstrapper is including the original values (called actual, I would rename to actual in this PR)

from sktime.transformations.bootstrap import STLBootstrapTransformer
from sktime.datasets import load_airline
from sktime.utils.plotting import plot_series  # doctest: +SKIP
y = load_airline()  # doctest: +SKIP
transformer = STLBootstrapTransformer(10)  # doctest: +SKI

y_hat = transformer.fit_transform(y)
                        Number of airline passengers
series_id   time_index                              
actual      1949-01                       112.000000
            1949-02                       118.000000
            1949-03                       132.000000
            1949-04                       129.000000
            1949-05                       121.000000
...                                              ...
synthetic_9 1960-08                       630.055938
            1960-09                       505.544319
            1960-10                       459.539478
            1960-11                       407.284782
            1960-12                       469.909948

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mhm.. I used the behavior of the STLBootstrap as template. This bootstrapper is including the original values (called actual, I would rename to actual in this PR)

Yes, but I think that is a special choice rather than user expected, and it complicates the interface. It is also not sth that tsbootstrap does (and imo it's the better design choice) - FYI @astrogilda

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will take a look on the other bootstrapped in sktime and check how they are behaving. But in that case, I think it would be good to streamline the behavior using deprecation cycles.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you are probably right. Safest perhaps is add a parameter include_actual or similar and switch its default.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we open an issue? I am guessing that all bootstrapping transformers are in the bootstrap folder.

f"synthetic_{i}"
for i in range(self.tsbootstrapper.config.n_bootstraps)
],
X.index,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this will break, since in genera the bootstrap samples are shorter.
I would leave range index if we cannot come up with sth better.

@fkiraly fkiraly changed the title [ENH] add tsboostrap adapter [ENH] tsbootstrap adapter Feb 18, 2024
@benHeid
Copy link
Contributor Author

benHeid commented Feb 24, 2024

How should we deal with this failure?

pyproject.toml Outdated Show resolved Hide resolved
@fkiraly fkiraly marked this pull request as ready for review March 20, 2024 21:04
@fkiraly
Copy link
Collaborator

fkiraly commented Mar 20, 2024

updated source after tsbootstrap 0.1.0 release

Copy link
Collaborator

@yarnabrina yarnabrina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Made few comments, only serious one is the __init__ one.

pyproject.toml Show resolved Hide resolved
sktime/transformations/bootstrap/_tsbootstrap.py Outdated Show resolved Hide resolved
sktime/transformations/bootstrap/_tsbootstrap.py Outdated Show resolved Hide resolved
sktime/transformations/bootstrap/_tsbootstrap.py Outdated Show resolved Hide resolved
"""
from sktime.utils.validation._dependencies import _check_soft_dependencies

deps = cls.get_class_tag("python_dependencies")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just observing it's probably being done directly against the package name in other adapters. But this may be better, as future changes in dependencies won't affect.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, we should do _check_estimator_deps(self)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

self is not available since get_test_params is a classmethod. However, we can instanitate a tsbootstrap for testing.. But I do not really like to do that.

Copy link
Collaborator

@fkiraly fkiraly Mar 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, then cls works too, as arg. No need to instantiate.

Copy link
Collaborator

@yarnabrina yarnabrina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My comments were addressed, thanks.

I don't understand the failed job. The status is failure, but it appears it's still running. I'm very confused, do you understand what's happening?

@fkiraly
Copy link
Collaborator

fkiraly commented Mar 22, 2024

Yes, that's weird. It displayed as still running, but GHA behaved as if it were failed. I restarted, and now it's gone.

@fkiraly fkiraly merged commit e25842e into main Mar 22, 2024
202 checks passed
@fkiraly fkiraly deleted the tsbootstrap_adapter branch March 22, 2024 18:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Adding new functionality module:transformations transformations module: time series transformation, feature extraction, pre-/post-processing
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[ENH] Adapter to TSBootstrap initial sktime integration
3 participants