Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] fix behaviour of FourierFeatures with pd.DatetimeIndex #3606

Merged
merged 9 commits into from Nov 10, 2022

Conversation

eenticott-shell
Copy link
Collaborator

Reference Issues/PRs

Fixes #3605

What does this implement/fix? Explain your changes.

Adds an extra case into FourierFeatures for dealing with a pd.DatetimeIndex. These are stored in nanoseconds and so the seasonalities have to also be specified in nanoseconds. Wanted to avoid this and work from frequency instead for the DatetimeIndex case.

What should a reviewer concentrate their feedback on?

Any other comments?

Copy link
Contributor

@ltsaprounis ltsaprounis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the bugfix! left a few comments, but nothing major.
I'm keen to see @fkiraly's thoughts on my comment here, before we merge. I either misinterpreted the functionality of the tag "enforce_index_type", or there's another bug in the tag behaviour.

sktime/transformations/series/fourier.py Outdated Show resolved Hide resolved
sktime/transformations/series/fourier.py Outdated Show resolved Hide resolved
sktime/transformations/series/fourier.py Outdated Show resolved Hide resolved
Copy link
Contributor

@ltsaprounis ltsaprounis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Based on this fix and on the discussion on #3605, I suggest we add pd.DateTimeIndex in the enforce_index_type list

@eenticott-shell
Copy link
Collaborator Author

So the real bug was in the enforce index! I've added pd.DatetimeIndex as an allowed type, and addressed the other changes you suggested. Also changed the logic when choosing the frequency to avoid so many nested if statements.

Copy link
Contributor

@ltsaprounis ltsaprounis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good!
I left a few suggestions in the comments.

sktime/transformations/series/fourier.py Outdated Show resolved Hide resolved
sktime/transformations/series/fourier.py Outdated Show resolved Hide resolved
sktime/transformations/series/fourier.py Outdated Show resolved Hide resolved
@ltsaprounis
Copy link
Contributor

Tests pass now!
One last thing, can you just add an extra test in the test_fourier.py with something similar to your example in the issue?

Just parametrise the test and add a Y_datetime.
If you specify a monthly freq, the outputs etc should be the same as in the existing test.

@ltsaprounis
Copy link
Contributor

@eenticott-shell - In the case where the index is datetime, will the transformed series index maintain the datetime index, or will it change to period? Cause I think the transform method might be changing the index permanently here. Can you quickly check and maybe add it in the test that the index is not changed. Something like:

from pandas.testing import assert_index_equal
assert_index_equal(left, right, exact=True)

@ltsaprounis
Copy link
Contributor

Perfect! that should be good to go now.
Will approve and merge once the CI is done.

@ltsaprounis
Copy link
Contributor

Will approve and merge once the CI is done.

@fkiraly - given it's a bug, can we merge this before the release? your call...

@eenticott-shell
Copy link
Collaborator Author

Thanks for your help! I think one of the tests might fail, I am having trouble running pytest locally but I am currently trying to fix!

@eenticott-shell
Copy link
Collaborator Author

Think that should have fixed it

Copy link
Contributor

@ltsaprounis ltsaprounis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the failing test is not related to your transformer:
FAILED sktime/transformations/panel/rocket/tests/test_Rocket.py::test_rocket_on_gunpoint - assert 0.9933333333333333 == 1.0

Left some comments for avoiding a potential side effect in the transform index and removing the redundant index copy in fit.

sktime/transformations/series/fourier.py Outdated Show resolved Hide resolved
sktime/transformations/series/fourier.py Outdated Show resolved Hide resolved
@fkiraly
Copy link
Collaborator

fkiraly commented Nov 4, 2022

@fkiraly - given it's a bug, can we merge this before the release? your call...

You need to approve and resolve conversations, @ltsaprounis, if that's what you advocate.

Copy link
Contributor

@GuzalBulatova GuzalBulatova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall, thank you @eenticott-shell ! Have only one question

sktime/transformations/series/fourier.py Show resolved Hide resolved
@GuzalBulatova
Copy link
Contributor

GuzalBulatova commented Nov 9, 2022

@ltsaprounis could you please resolve the pending changes if you're satisfied with them?

Copy link
Contributor

@ltsaprounis ltsaprounis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good! thanks for the fix and your patience with the change requests @eenticott-shell! Will merge once the tests are done.

@ltsaprounis ltsaprounis merged commit 274ea75 into sktime:main Nov 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] FourierFeatures with pd.DatetimeIndex
4 participants