Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DEPR: Period[B] #53446

Closed
3 tasks done
jbrockmendel opened this issue May 29, 2023 · 10 comments · Fixed by #53511
Closed
3 tasks done

DEPR: Period[B] #53446

jbrockmendel opened this issue May 29, 2023 · 10 comments · Fixed by #53511
Labels
Deprecate Functionality to remove in pandas Needs Discussion Requires discussion from core team before further action Period Period data type

Comments

@jbrockmendel
Copy link
Member

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

NA

Issue Description

TL;DR: can we deprecate BDay Period(|Dtype|Index|Array) and tell users to use a DatetimeIndex with freq="B" instead?

Context: lots of user confusion has resulted from a) using "freq" for PeriodX to mean something different than it does for DatetimeIndex (#51256, #38885, #47227) and b) using DateOffsets to define PeriodX (#5091, #13871, #38914, #38859).

The option I'm leaning toward to address this is to change/deprecate PeriodX.freq to "unit" (the attribute, the constructor keywords, and asfreq->as_unit). The sticking point I'm finding is in finding a replacement for the current usage period + period.freq * N. For everything except BDay, we could tell users to do something like period + np.timedelta64(N, period.unit) instead (actually just period + N works, but I'd actually prefer to deprecate that).

So I'm wondering if we really need to support Period[B]. Anyone have a strong opinion?

Expected Behavior

NA

Installed Versions

Replace this line with the output of pd.show_versions()

@jbrockmendel jbrockmendel added Bug Needs Triage Issue that has not been reviewed by a pandas team member Period Period data type Deprecate Functionality to remove in pandas Needs Discussion Requires discussion from core team before further action and removed Bug Needs Triage Issue that has not been reviewed by a pandas team member labels May 29, 2023
@jbrockmendel jbrockmendel mentioned this issue Jun 3, 2023
5 tasks
@MarcoGorelli
Copy link
Member

Reproducible Example
NA

Any chance you could include an example of what you'd like to deprecate, and what users should do instead please?

@jbrockmendel
Copy link
Member Author

Any chance you could include an example of what you'd like to deprecate, and what users should do instead please?

pd.anything(..., dtype="Period[B]")
pd.period_range(..., freq="B")
pd.PeriodIndex(..., freq="B")
pd.Period(..., freq="B")

For the PeriodIndex cases I'd advise users to use a DatetimeIndex with freq="B". For the a user would be out of luck.

@MarcoGorelli
Copy link
Member

sounds fine, thanks!

@torext
Copy link
Contributor

torext commented Sep 6, 2023

This now seems to be causing a lot of warnings when simply calling .plot() on a DataFrame with freq="B" DatetimeIndex:

Failed to convert value(s) to axis units: (datetime.date(1970, 1, 1), datetime.date(1970, 1, 2))
  File "/home/torext/miniconda3/envs/testenv/lib/python3.11/site-packages/matplotlib/axis.py", line 1732, in convert_units
    ret = self.converter.convert(x, self.units, self)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/torext/miniconda3/envs/testenv/lib/python3.11/site-packages/pandas/plotting/_matplotlib/converter.py", line 238, in convert
    values = PeriodConverter._convert_1d(values, units, axis)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/torext/miniconda3/envs/testenv/lib/python3.11/site-packages/pandas/plotting/_matplotlib/converter.py", line 257, in _convert_1d
    return [get_datevalue(x, axis.freq) for x in values]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/torext/miniconda3/envs/testenv/lib/python3.11/site-packages/pandas/plotting/_matplotlib/converter.py", line 257, in <listcomp>
    return [get_datevalue(x, axis.freq) for x in values]
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/torext/miniconda3/envs/testenv/lib/python3.11/site-packages/pandas/plotting/_matplotlib/converter.py", line 265, in get_datevalue
    return Period(date, freq).ordinal
           ^^^^^^^^^^^^^^^^^^
  File "/home/torext/miniconda3/envs/testenv/lib/python3.11/site-packages/period.pyx", line 2823, in pandas._libs.tslibs.period.Period.__new__
FutureWarning: Period with BDay freq is deprecated and will be removed in a future version. Use a DatetimeIndex with BDay freq instead.

The above exception was the direct cause of the following exception:

Any chance this can be fixed? I (guess I) understand deprecating this use of the Period object, but if it mostly just affects internals, which when called normally by the user work but now throw a bunch of warnings the user can do nothing about, then it's quite annoying...

@mroeschke
Copy link
Member

@jbrockmendel do you remember if there were any challenges with silencing this warnings for the plotting code?

@torext
Copy link
Contributor

torext commented Sep 7, 2023

Also, what's curious is that running my script throws with

FutureWarning: PeriodDtype[B] is deprecated and will be removed in a future version. Use a DatetimeIndex with freq='B' instead

even though I added the following at the very start of my entry point:

warnings.simplefilter("error")
warnings.filterwarnings("ignore", "Period with BDay freq is deprecated", FutureWarning)
warnings.filterwarnings(
    "ignore", "PeriodDtype[B] is deprecated and will be removed", FutureWarning
)

I.e. that second filter I added does override the very first simplefilter, but the very last filter has no effect.

@jbrockmendel
Copy link
Member Author

@jbrockmendel do you remember if there were any challenges with silencing this warnings for the plotting code?

IIIRC it was as normal filterwarnings without any complications. There's a note-to-self that we need a better solution before the deprecation is enforced.

@torext
Copy link
Contributor

torext commented Sep 13, 2023

A very simple repro showing the warnings happening is as follows:

import warnings
import pandas._testing as tm

warnings.simplefilter("error")
_ = tm.makeTimeDataFrame().plot()

For me this throws even with the latest checkout of the repo. In fact, the first test in pandas/tests/plotting/frame/test_frame.py shows the same warnings.

There's a note-to-self that we need a better solution before the deprecation is enforced.

Stepping through the plotting code it looks to me like that place where this note was put is not sufficient to capture all instances of the warning: it silences the warning that would happen on line 1363 of pandas/plotting/_matplotlib/core.py, but then the warning gets triggered again on line 1392 of the same module.

Given the warning should be suppressed for the most naive and simple use of plotting functionality, may I suggest the warning filter get moved way higher up the hierarchy, potentially all the way to the __call__ method of the PlotAccessor even? That way any more specialised/direct use of the Period objects doesn't get the warnings filtered, but simple users like me aren't left scratching their head 🙂

@jbrockmendel
Copy link
Member Author

may I suggest the warning filter get moved way higher up the hierarchy

I'd be open to that, but first I'd like to try getting the warning suppression closer to the source of the problem. I can get the example you posted to run by catching warnings in PeriodConverter.convert_1d and TimeSeries_DateFormatter.__call__. I suspect catching in _daily_finder might also be necessary for some other cases.

PR would be welcome.

@torext
Copy link
Contributor

torext commented Sep 14, 2023

PR would be welcome.

@jbrockmendel I submitted a PR based on your suggestions: #55138. I still feel that a single filter at the level of, say, PlotAccessor.__call__ would be more conservative and would have the advantage of just one single location where the filter is placed, as opposed to now having these multiple (I think 4?) invocations of what is essentially the same filter.

maxwell-k added a commit to maxwell-k/qgridtrusted that referenced this issue Jan 12, 2024
Before this change test_period_object_column fails with the message
below:

    FAILED qgrid/tests/test_grid.py::test_period_object_column - OverflowError: Maximum recursion level reached

After this change the test passes.

See also
pandas-dev/pandas#53446
maxwell-k added a commit to maxwell-k/qgridtrusted that referenced this issue Jan 12, 2024
Before this change test_period_object_column fails with the message
below:

    FAILED qgrid/tests/test_grid.py::test_period_object_column - OverflowError: Maximum recursion level reached

After this change the test passes.

See also
pandas-dev/pandas#53446
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Deprecate Functionality to remove in pandas Needs Discussion Requires discussion from core team before further action Period Period data type
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants