Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

API: DataFrame(list_with_ea) #49593

Open
jbrockmendel opened this issue Nov 9, 2022 · 0 comments
Open

API: DataFrame(list_with_ea) #49593

jbrockmendel opened this issue Nov 9, 2022 · 0 comments
Labels
Constructors Series/DataFrame/Index/pd.array Constructors

Comments

@jbrockmendel
Copy link
Member

jbrockmendel commented Nov 9, 2022

DataFrame(nested_list) generally treats each list entry as a row. The exception (until #49592) is if the first element is a Categorical in which we treated elements as columns. We have zero tests passing a list[EA] except when the first entry is a Categorical, which suggests to me that the behavior here has not gotten much attention.

For non-EA and some EAs (Period, dt64tz, interval) inference basically works right. But for Categorical, pyarrow, and presumably 3rd-party EAs it doesn't:

df = pd.DataFrame(np.arange(9).reshape(3, 3)).astype("int64[pyarrow]")
arrs = [df[i]._values for i in range(3)]
pd.DataFrame(arrs).dtypes  # <- back to numpy dtypes

We could reasonably use concat-like logic for dtype inference. (or for EAs that support 2D could just concat directly).

Another approach that might work is to do something like DataFrame({i: rows[i] for i in range(len(rows))}).T. That works in the pyarrow example above.

@jbrockmendel jbrockmendel added Bug Needs Triage Issue that has not been reviewed by a pandas team member Constructors Series/DataFrame/Index/pd.array Constructors and removed Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Nov 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Constructors Series/DataFrame/Index/pd.array Constructors
Projects
None yet
Development

No branches or pull requests

1 participant