[ENH] Add hour_of_week option to DateTimeFeatures transformer

[VyomkeshVyas]

What does this implement/fix? Explain your changes.

This PR adds hour_of_week as an option in the manual_section argument in the DateTimeFeatures transformer.
hour_of_week could be used as an alternate to the hour_of_day and day_of_week combined.

Example usage :

import pandas as pd
from sktime.transformations.series.date import DateTimeFeatures

y = pd.DataFrame(
    data={"y": range(6)},
    index=pd.date_range(start="2023-01-01", freq="H", periods=6),
) 
transformer = DateTimeFeatures(
    manual_selection=["hour_of_week"], keep_original_columns=True
)

yt = transformer.fit_transform(y)

What should a reviewer concentrate their feedback on?

Around three lines have been added in the existing code to calculate the hour_of_week (under the comprehensive feature_scope), aligning with the current code structure.
I'm not entirely sure about the utility of fourier column in the DUMMIES dataframe (computed in _prep_dummies function) and its role in calculating the final datetime features.

Did you add any tests for the change?

I've added a unit test in the existing tests test_date.py.

PR checklist

For all contributions

• I've added a unit test and made sure it pass locally.
• The PR title starts with either [ENH], [MNT], [DOC], or [BUG]. [BUG] - bugfix, [MNT] - CI, test framework, [ENH] - adding or improving code, [DOC] - writing or improving documentation or docstrings.

[fkiraly]

Great contribution!

We should check with @KishManani (current owner of the estimator) whether this addition is fine.

[fkiraly]

Thanks!

Remarks only from maintenance perspective:

• @KishManani is the current owner so should review and approve
• Adding this to the "comprehensive" feature set directly will return additional columns to users currently using that pre-defined feature set, so it may break code somewhere. We will hence need to follow deprecation policy, earliest we can switch it in is 0.21.0. Can we add it and not have it part of any of the pre-defined feature sets for now, and add a warning that this will expand in 0.21.0? Alternatively, just not add it at all.

[KishManani]

LGTM.

Out of curiosity, have you found this feature useful in practice? What was the use case that drove you to use it?

I'm wondering where we want to draw the line in terms of adding more features in here (e.g., in theory we could add minute of week, second of week, etc.) but these are unlikely to be helpful I would have thought.

[fkiraly]

Great! We now just have to deal with the deprecation/change (or not putting it in the feature set).

[fkiraly]

@KishManani, @VyomkeshVyas, does on of you perchance know whether a feature can be added that does not belong to any of the pre-defined feature sets?

For change policy, the easiest options would be to:

• keep it as not part of a feature set (if it is possible to have it outside)
• or raise a warning whenever "comprehensive" is selected at init, that from 0.22.0 it will contain this new feature.

[VyomkeshVyas]

LGTM.

Out of curiosity, have you found this feature useful in practice? What was the use case that drove you to use it?

I'm wondering where we want to draw the line in terms of adding more features in here (e.g., in theory we could add minute of week, second of week, etc.) but these are unlikely to be helpful I would have thought.

I came across this idea while working on a price forecasting task for data having hourly granularity. I can't generalize the results but, with a simple linear regression model there was a slight increase (0.10) in RMSE by replacing hour_of_day and day_of_week feature with hour_of_week, while a slight decrease (0.06) in RMSE with XG Boost model.

I think theoretically there are many possible combinations of generating features and it wouldn't be worthwhile to include every single of those, but as suggested by @fkiraly, Seasonal Remainder feature extractor is a good option, a user will be able to create any possible combination of features as per the requirement.

[fkiraly]

@VyomkeshVyas, have you thought about your preference regarding the warning/deprecation procedure?

[VyomkeshVyas]

@VyomkeshVyas, have you thought about your preference regarding the warning/deprecation procedure?

@fkiraly I think "raise a warning whenever "comprehensive" is selected at init, that from 0.22.0 it will contain this new feature" is a good option as keeping the feature outside the pre-defined feature sets will itself create a new feature set with just "hour_of_week" as feature, which might not make sense to a user.

[fkiraly]

agreed - do you want me to add the warnings, or is it clear where/how?

[fkiraly]

@VyomkeshVyas, do you want me to add the warnings, or do you want to do this?

This is nearly done, let's try to get it into 0.20.1! Happy either way, just let us know.

[fkiraly]

Thanks! Good to go now.

(FYI, I added a reminder comment for 0.22.0)