New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BUG] fix BoxCoxTransformer
failure after scipy
1.11.0
#4770
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
fkiraly
added
module:transformations
transformations module: time series transformation, feature extraction, pre-/post-processing
bugfix
Fixes a known bug or removes unintended behavior
labels
Jun 25, 2023
update: just learnt that an initial bracket need not contain bounds - so I simply set the bracket to bounds if provided, while passing bounds if provided also to the optimizer (constrained) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
bugfix
Fixes a known bug or removes unintended behavior
module:transformations
transformations module: time series transformation, feature extraction, pre-/post-processing
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #4769 by using
scipy
's on-board box-cox-fitter for methods"mle"
and"pearsonr"
and enforcing positive values.The reason for the failure was use of negative values in some interface test cases, which previously would produce garbage results but
scipy
would not complain. As of 1.11.0, it complains (albeit with a confusing error message that makes it hard to track down that it is in fact from negative values in some test cases). See further discussion here: scipy/scipy#18761This is now solved by introducing an
enforce_positive
argument toBoxCoxTransformer
, which ensures positive values before fitting. This can be turned off to obtain the previous behaviour, but for positive values it will be the same (although with an additionalnp.sign
andnp.abs
calculation that should be fast).Also makes the following changes, while we're cleaning this up:
sktime
box-cox-fitter inBoxCoxTransformer
was not usingscipy
's since the latter did not have settable bounds when the former was originally implemented. I have hence replaced it withscipy
's own box-cox-fitter and using a bounded optimizer if bounds are passed, fromscipy
1.7.0 on where it exists in the form we need. Pre-1.7.0, I've left the old behaviour.scipy
"fixed"
method, which allows to use the transformer with a fixed parameter - or tune it via grid search.method
Note: the fix is behaviour changing, but imo does not require deprecation because it changes some behaviour from buggy/misleading (only in the sub-case where negative values were passed) to sensible.