We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Creating a pandas DataFrame using ArrowDtype holding a pyarrow array with extension type fails using pandas 2.0.3:
ArrowDtype
In [1]: ext_arr = pa.array(pd.period_range("2012", periods=3)) In [2]: df = pa.table({'col': ext_arr}).to_pandas(types_mapper=pd.ArrowDtype) --------------------------------------------------------------------------- NotImplementedError Traceback (most recent call last) Cell In[2], line 1 ----> 1 df = pa.table({'col': ext_arr}).to_pandas(types_mapper=pd.ArrowDtype) ... File ~/miniconda3/envs/pandas20/lib/python3.11/site-packages/pandas/core/internals/api.py:73, in make_block(values, placement, klass, ndim, dtype) 70 placement = BlockPlacement(placement) 72 ndim = maybe_infer_ndim(values, placement, ndim) ---> 73 if is_datetime64tz_dtype(values.dtype) or is_period_dtype(values.dtype): 74 # GH#41168 ensure we can pass 1D dt64tz values 75 # More generally, any EA dtype that isn't is_1d_only_ea_dtype 76 values = extract_array(values, extract_numpy=True) 77 values = ensure_block_shape(values, ndim) File ~/miniconda3/envs/pandas20/lib/python3.11/site-packages/pandas/core/dtypes/common.py:415, in is_period_dtype(arr_or_dtype) 387 """ 388 Check whether an array-like or dtype is of the Period dtype. 389 (...) 411 True 412 """ 413 if isinstance(arr_or_dtype, ExtensionDtype): 414 # GH#33400 fastpath for dtype object --> 415 return arr_or_dtype.type is Period 417 if arr_or_dtype is None: 418 return False File ~/miniconda3/envs/pandas20/lib/python3.11/site-packages/pandas/core/arrays/arrow/dtype.py:150, in ArrowDtype.type(self) 148 return type(pa_type) 149 else: --> 150 raise NotImplementedError(pa_type) NotImplementedError: extension<pandas.period<ArrowPeriodType>>
This works on the main branch, although there are still some other issues such as the repr not working, see #54062
Installed versions:
In [5]: pd.show_versions() /home/joris/miniconda3/envs/pandas20/lib/python3.11/site-packages/_distutils_hack/__init__.py:33: UserWarning: Setuptools is replacing distutils. warnings.warn("Setuptools is replacing distutils.") INSTALLED VERSIONS ------------------ commit : 0f437949513225922d851e9581723d82120684a6 python : 3.11.3.final.0 python-bits : 64 OS : Linux OS-release : 5.4.0-152-generic Version : #169-Ubuntu SMP Tue Jun 6 22:23:09 UTC 2023 machine : x86_64 processor : x86_64 byteorder : little LC_ALL : None LANG : en_US.UTF-8 LOCALE : en_US.UTF-8 pandas : 2.0.3 numpy : 1.24.2 pytz : 2023.3 dateutil : 2.8.2 setuptools : 67.6.1 pip : 23.0.1 Cython : None pytest : None hypothesis : None sphinx : None blosc : None feather : None xlsxwriter : None lxml.etree : None html5lib : None pymysql : None psycopg2 : None jinja2 : None IPython : 8.12.0 pandas_datareader: None bs4 : None bottleneck : None brotli : None fastparquet : None fsspec : None gcsfs : None matplotlib : None numba : None numexpr : None odfpy : None openpyxl : None pandas_gbq : None pyarrow : 11.0.0 pyreadstat : None pyxlsb : None s3fs : None scipy : None snappy : None sqlalchemy : None tables : None tabulate : None xarray : None xlrd : None zstandard : None tzdata : 2023.3 qtpy : None pyqt5 : None
The text was updated successfully, but these errors were encountered:
I think this was closed by #54130
Sorry, something went wrong.
No branches or pull requests
Creating a pandas DataFrame using
ArrowDtype
holding a pyarrow array with extension type fails using pandas 2.0.3:This works on the main branch, although there are still some other issues such as the repr not working, see #54062
Installed versions:
The text was updated successfully, but these errors were encountered: