Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Python] Support halffloat for Arrow list to pandas #36168

Closed
izveigor opened this issue Jun 19, 2023 · 0 comments · Fixed by #35944
Closed

[Python] Support halffloat for Arrow list to pandas #36168

izveigor opened this issue Jun 19, 2023 · 0 comments · Fixed by #35944
Assignees
@izveigor
Labels
Component: Python Type: bug
Milestone
13.0.0

Comments

@izveigor
Copy link
Contributor

Describe the bug, including details regarding any error messages, version, and platform.

When we translate float16 from pyarrow to pandas, no error occurs:

>>> df = pd.DataFrame({"a": [np.float16(1), np.float16(2), np.float16(3)]})
>>> table = pa.Table.from_pandas(df)
>>> df_new = table.to_pandas()

But, when we use list:

>>> df = pd.DataFrame({"a": [[np.float16(1)], [np.float16(2)], [np.float16(3)]]})
>>> table = pa.Table.from_pandas(df)
>>> df_new = table.to_pandas()
pyarrow.lib.ArrowNotImplementedError: Not implemented type for Arrow list to pandas: halffloat

Component(s)

Python
@github-actions github-actions bot mentioned this issue Jun 19, 2023
Merged
jorisvandenbossche pushed a commit that referenced this issue Jun 28, 2023
@izveigor
GH-36168: [C++][Python] Support halffloat for Arrow list to pandas (#…
e5de6a5 
…35944)

### Rationale for this change
When we translate `float16` from `pyarrow` to `pandas`, no error occurs:
```
>>> df = pd.DataFrame({"a": [np.float16(1), np.float16(2), np.float16(3)]})
>>> table = pa.Table.from_pandas(df)
>>> df_new = table.to_pandas()
```
But, when we use list:
```
>>> df = pd.DataFrame({"a": [[np.float16(1)], [np.float16(2)], [np.float16(3)]]})
>>> table = pa.Table.from_pandas(df)
>>> df_new = table.to_pandas()
pyarrow.lib.ArrowNotImplementedError: Not implemented type for Arrow list to pandas: halffloat
```

### What changes are included in this PR?

### Are these changes tested?
Yes

### Are there any user-facing changes?
Yes

* Closes: #36168

Authored-by: izveigor <izveigor@gmail.com>
Signed-off-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
@jorisvandenbossche jorisvandenbossche added this to the 13.0.0 milestone Jun 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@izveigor izveigor

Labels
Component: Python Type: bug
Projects
None yet
Milestone
13.0.0
Development

Successfully merging a pull request may close this issue.

GH-36168: [C++][Python] Support halffloat for Arrow list to pandas
2 participants
@jorisvandenbossche @izveigor