Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] pandas 2 compatibility issues #4426

Closed
fkiraly opened this issue Apr 3, 2023 · 2 comments · Fixed by #4427
Closed

[BUG] pandas 2 compatibility issues #4426

fkiraly opened this issue Apr 3, 2023 · 2 comments · Fixed by #4427
Labels
bug Something isn't working maintenance Continuous integration, unit testing & package distribution

Comments

@fkiraly
Copy link
Collaborator

fkiraly commented Apr 3, 2023 •
edited
Loading

Some problems with pandas 2 as released on Apr 3:

  • ForecastingHorizon to_absolute no longer seems to work with pandas 2 as released, due to incomaptibility with the index based logic in to_absolute
  • pd.DataFrame constructor no longer accepts ForecastingHorizon as index or columns argument
  • conversion from nested_univ to pd-multiindex causes pd.Index indices which are not compatible with data input contracts
  • changes that make the contract of DataFrame.agg more restrictive and cause exceptions
  • loc can no longer be used to write to index locations that do not exist
  • the pandas.util.testing module no longer exists, contents moved to pandas.testing
  • Timestamp.freq is now removed. Despite dealing with this already, there were still some instances in sktime using this.
  • slicing timelike indices with a single index of lower granularity is no longer possible, e.g., df["1950"] on monthly data now raises error instead of returning all months in 1950

Further problems to be mapped by #4430
@fkiraly fkiraly added bug Something isn't working maintenance Continuous integration, unit testing & package distribution labels Apr 3, 2023
@fkiraly fkiraly mentioned this issue Apr 3, 2023
Merged
fkiraly added a commit that referenced this issue Apr 3, 2023
@fkiraly
[MNT] reintroduce pandas<2.0.0 bound (#4428)
7f23b01 
Reintroduces the `pandas` 2 bound given compatibility issues, see #4426
@fkiraly fkiraly mentioned this issue Apr 3, 2023
Merged
@fkiraly
Copy link
Collaborator Author

fkiraly commented Apr 3, 2023

why was this not detected?

It appears that some of the soft dependenies imply pandas < 2.0.0, so relaxing the bound in sktime is not enough if all the soft dependencies are present.

@fkiraly
Copy link
Collaborator Author

fkiraly commented Apr 4, 2023 •
edited
Loading

code to trigger/diagnose check failure:

from sktime.transformations.series.exponent import ExponentTransformer
from sktime.utils._testing.panel import _make_panel_X

X = _make_panel_X()
t = ExponentTransformer()
t.fit_transform(X)

fkiraly added a commit that referenced this issue Apr 10, 2023
@fkiraly
[MNT] pandas 2 compatibility fixes (#4427)
d0d8298 
Fixes #4426.
This PR contains some tentative fixes for compatibility issues arising from the `pandas` 2 release on Apr 3:

* fixes to index handling in `ForecastingHorizon.to_absolute`
* fixes to many instances of using `ForecastingHorizon` as `pd.DataFrame` `index` in constructor or setter (`.index = ...`), this is no longer possible
* writing to a `loc` index that does not exist no longer creates it but raises an exception - this was replaced by ensuring the right rows and columns are present before the write - `AutoETS`, `UnobservedComponents`
* replace deprecated `is_integer` and `is_numeric` with recommended replacement calls to `pandas.api.types`
* fixed old `from_nested_to_multi_index` utility producing `object` valued multiindex (instead of integer typed etc)
* same for `_convert_tsf_to_hierarchical` utility in data loaders
* replace deprecated `pandas.util.testing` imports with `pandas.testing`
* #4435
* #4436
* replace in-place nested pandas operations with construction of new frame: `test_interpolate`
* #4438
* replace single-index temporal slicing at lower granularity with slice-based slicing

Relies on #4429 for testing.

Does *not* change the dependency bounds.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working maintenance Continuous integration, unit testing & package distribution
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[MNT] pandas 2 compatibility fixes sktime/sktime
1 participant
@fkiraly