BUG: to_pydatetime() -> 'pyarrow.lib.DataType' object has no attribute 'unit'

[s-banach]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import polars as pl
import datetime

pl.Series([datetime.date(2022,12,31)]).to_pandas(use_pyarrow_extension_array=True).dt.to_pydatetime()

"""AttributeError: 'pyarrow.lib.DataType' object has no attribute 'unit'"""

Issue Description

For context, I was feeding a pandas DataFrame with pyarrow dtypes into the dash package, which called s.dt.to_pydatetime() on a date series s. Maybe the problem is dash's fault, but this seems like a pandas bug.

Expected Behavior

Return datetime objects as expected.

Installed Versions

ModuleNotFoundError: No module named 'PySide6'

[mroeschke]

Can you post the pull traceback?

[s-banach]

~/.local/lib/python3.9/site-packages/pandas/core/indexes/accessors.py in to_pydatetime(self)
    211 
    212     def to_pydatetime(self):
--> 213         return cast(ArrowExtensionArray, self._parent.array)._dt_to_pydatetime()
    214 
    215     def isocalendar(self):

~/.local/lib/python3.9/site-packages/pandas/core/arrays/arrow/array.py in _dt_to_pydatetime(self)
   2085     def _dt_to_pydatetime(self):
   2086         data = self._data.to_pylist()
-> 2087         if self._dtype.pyarrow_dtype.unit == "ns":
   2088             data = [None if ts is None else ts.to_pydatetime(warn=False) for ts in data]
   2089         return np.array(data, dtype=object)

AttributeError: 'pyarrow.lib.DataType' object has no attribute 'unit'

[mroeschke]

Okay here's a pandas reproducer

In [1]: import datetime

In [2]: import numpy as np; import pandas as pd; import pyarrow as pa

In [3]: pd.Series(pd.arrays.ArrowExtensionArray(pa.array([datetime.date.today()]))).dt.to_pydatetime()
AttributeError: 'pyarrow.lib.DataType' object has no attribute 'unit'

Not 100% that this should work, datetime.date and datetime.datetime are different objects. Probably should raise a better error message here that the pyarrow type should be pa.timestamp type specifically

[jbrockmendel]

isnt this why we have get_unit_from_pa_dtype?

[mroeschke]

get_unit_from_pa_dtype

Looks specifically used for pa.time types, but for a totally different application.

The issue here is date[unit][pyarrow] (and not necessarily timestamp[unit][pyarrow]) type working with a to_pydatetime() method

[phofl]

Labelling 2.0.1 for now, think it might make sense back porting a better error message