Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Python] Converting empty table to batch results in empty list #37200

Closed
AlenkaF opened this issue Aug 16, 2023 · 1 comment
Closed

[Python] Converting empty table to batch results in empty list #37200

AlenkaF opened this issue Aug 16, 2023 · 1 comment

Comments

@AlenkaF
Copy link
Member

AlenkaF commented Aug 16, 2023

Describe the bug, including details regarding any error messages, version, and platform.

When converting an empty table with a known schema to a record batch, the information about the schema is lost and the result is an empty list:

>>> my_schema = pa.schema([pa.field('col1', pa.int64())])
>>> df = pa.table([[]], schema=my_schema)
>>> df
pyarrow.Table
col1: int64
----
col1: [[]]
>>> df.to_batches()
[]

but in comparison, if we are creating an empty batch with a known schema we get the following:

>>> batch = pa.record_batch([[]], schema=my_schema)
>>> batch
pyarrow.RecordBatch
col1: int64
----
col1: []

and I think this should be same for the case of converting an empty table with known schema to a record batch.

Component(s)

Python

@AlenkaF
Copy link
Member Author

AlenkaF commented Oct 5, 2023

Closing this as there is not a clear idea what to do in cases of empty tables with (can be multiple) empty chunks. Can reopen in future if we see the need to do so.

@AlenkaF AlenkaF closed this as not planned Won't fix, can't repro, duplicate, stale Oct 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant