BUG: Series.describe treating pyarrow timestamps/timedeltas as categorical

[lukemanley]

• Tests added and passed if fixing a bug or adding a new feature
• All code checks passed.
• Added type annotations to new arguments/methods/functions.
• Added an entry in the latest doc/source/whatsnew/v2.0.2.rst file if fixing a bug or adding a new feature.

Fixes the following such that pyarrow-backed timestamps and timedeltas are described in the same way numpy-backed timestamps and timedeltas are described.

Current behavior:

In [1]: import pandas as pd

In [2]: df_numpy = pd.DataFrame({
   ...:     "datetime": pd.to_datetime(range(10)),
   ...:     "timedelta": pd.to_timedelta(range(10)),
   ...:     "numeric": range(10),
   ...: })

In [3]: df_pyarrow = df_numpy.convert_dtypes(dtype_backend="pyarrow")

In [4]: df_numpy.describe()
Out[4]: 
                            datetime                  timedelta   numeric
count                             10                         10  10.00000
mean   1970-01-01 00:00:00.000000004  0 days 00:00:00.000000004   4.50000
min              1970-01-01 00:00:00            0 days 00:00:00   0.00000
25%    1970-01-01 00:00:00.000000002  0 days 00:00:00.000000002   2.25000
50%    1970-01-01 00:00:00.000000004  0 days 00:00:00.000000004   4.50000
75%    1970-01-01 00:00:00.000000006  0 days 00:00:00.000000006   6.75000
max    1970-01-01 00:00:00.000000009  0 days 00:00:00.000000009   9.00000
std                              NaN  0 days 00:00:00.000000003   3.02765

In [5]: df_pyarrow.describe()
Out[5]: 
                   datetime        timedelta  numeric
count                    10               10     10.0
unique                   10               10     <NA>
top     1970-01-01 00:00:00  0 days 00:00:00     <NA>
freq                      1                1     <NA>
mean                    NaN              NaN      4.5
std                     NaN              NaN  3.02765
min                     NaN              NaN      0.0
25%                     NaN              NaN     2.25
50%                     NaN              NaN      4.5
75%                     NaN              NaN     6.75
max                     NaN              NaN      9.0

New behavior:

In [4]: df_pyarrow.describe()
Out[4]: 
                            datetime                  timedelta   numeric
count                             10                         10  10.00000
mean   1970-01-01 00:00:00.000000004  0 days 00:00:00.000000004   4.50000
min              1970-01-01 00:00:00            0 days 00:00:00   0.00000
25%    1970-01-01 00:00:00.000000002  0 days 00:00:00.000000002   2.25000
50%    1970-01-01 00:00:00.000000004  0 days 00:00:00.000000004   4.50000
75%    1970-01-01 00:00:00.000000006  0 days 00:00:00.000000006   6.75000
max    1970-01-01 00:00:00.000000009  0 days 00:00:00.000000009   9.00000
std                              NaN  0 days 00:00:00.000000003   3.02765

[phofl]

Could you do this explicitly? I might be missing something...

Edit: When creating the Series

[lukemanley]

sure - updated

[mroeschke]

Nice catch! Thanks @lukemanley

[lumberbot-app]

Owee, I'm MrMeeseeks, Look at me.

There seem to be a conflict, please backport manually. Here are approximate instructions:

1. Checkout backport branch and update it.

git checkout 2.0.x
git pull

2. Cherry pick the first parent branch of the this PR on top of the older branch:

git cherry-pick -x -m1 9442dfcc4f87c68632eb3ffbaf8dae6da4b532bd

3. You will likely have some merge/cherry-pick conflict here, fix them and commit:

git commit -am 'Backport PR #53001: BUG: Series.describe treating pyarrow timestamps/timedeltas as categorical'

4. Push to a named branch:

git push YOURFORK 2.0.x:auto-backport-of-pr-53001-on-2.0.x

5. Create a PR against branch 2.0.x, I would have named this PR:

"Backport PR #53001 on branch 2.0.x (BUG: Series.describe treating pyarrow timestamps/timedeltas as categorical)"

And apply the correct labels and milestones.

Congratulations — you did some good work! Hopefully your backport PR will be tested by the continuous integration and merged soon!

Remember to remove the Still Needs Manual Backport label once the PR gets merged.

If these instructions are inaccurate, feel free to suggest an improvement.

[mroeschke]

Mind backporting @lukemanley?

[lukemanley]

Mind backporting @lukemanley?

just backported in #53031