Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Python][C#] Pyarrow allows null column names but Apache.Arrow for C# doesn't #36588

Closed
DanTm99 opened this issue Jul 10, 2023 · 1 comment · Fixed by #39167
Closed

[Python][C#] Pyarrow allows null column names but Apache.Arrow for C# doesn't #36588

DanTm99 opened this issue Jul 10, 2023 · 1 comment · Fixed by #39167
Assignees
@CurtHagenlocher
Labels
Component: C# Component: Python Type: bug
Milestone
15.0.0

Comments

@DanTm99
Copy link
Contributor

DanTm99 commented Jul 10, 2023 •
edited
Loading

Describe the bug, including details regarding any error messages, version, and platform.

Using serialize_pandas() in pyarrow to serialize a pandas dataframe to arrow bytes allows for columns with a null name.

However, attempting to deserialize these bytes in the Apache Arrow C# library results in an ArgumentNullException being thrown from the Field constructor with the message Value cannot be null. (Parameter 'name').

If null field names are allowed in the Apache Arrow spec, then this is a bug in the C# library. If they are not allowed, then would this be a bug in pyarrow?

Component(s)

C#, Python
@CurtHagenlocher CurtHagenlocher mentioned this issue Jul 17, 2023
Merged
@DanTm99
Copy link
Contributor Author

DanTm99 commented Aug 9, 2023 •
edited
Loading

Related issue in C# library: #32886

@github-actions github-actions bot mentioned this issue Dec 10, 2023
Merged
CurtHagenlocher added a commit that referenced this issue Dec 11, 2023
@CurtHagenlocher
GH-36588: [C#] Support blank column names and enable more integration…
c87faf8 
… tests. (#39167)

### What changes are included in this PR?

Allows field names to be blank (but not null).
Enables schema metadata to be read from JSON integration tests.
Enables integration tests for cases that are now working.

### Are these changes tested?

Yes.

* Closes: #36588

Authored-by: Curt Hagenlocher <curt@hagenlocher.org>
Signed-off-by: Curt Hagenlocher <curt@hagenlocher.org>
@CurtHagenlocher CurtHagenlocher added this to the 15.0.0 milestone Dec 11, 2023
mapleFU pushed a commit to mapleFU/arrow that referenced this issue Dec 13, 2023
@CurtHagenlocher @mapleFU
apacheGH-36588: [C#] Support blank column names and enable more integ…
dedee2d 
…ration tests. (apache#39167)

### What changes are included in this PR?

Allows field names to be blank (but not null).
Enables schema metadata to be read from JSON integration tests.
Enables integration tests for cases that are now working.

### Are these changes tested?

Yes.

* Closes: apache#36588

Authored-by: Curt Hagenlocher <curt@hagenlocher.org>
Signed-off-by: Curt Hagenlocher <curt@hagenlocher.org>
clayburn pushed a commit to clayburn/arrow that referenced this issue Jan 23, 2024
@CurtHagenlocher @clayburn
apacheGH-36588: [C#] Support blank column names and enable more integ…
40eb1c4 
…ration tests. (apache#39167)

### What changes are included in this PR?

Allows field names to be blank (but not null).
Enables schema metadata to be read from JSON integration tests.
Enables integration tests for cases that are now working.

### Are these changes tested?

Yes.

* Closes: apache#36588

Authored-by: Curt Hagenlocher <curt@hagenlocher.org>
Signed-off-by: Curt Hagenlocher <curt@hagenlocher.org>
dgreiss pushed a commit to dgreiss/arrow that referenced this issue Feb 19, 2024
@CurtHagenlocher @dgreiss
apacheGH-36588: [C#] Support blank column names and enable more integ…
e8e914d 
…ration tests. (apache#39167)

### What changes are included in this PR?

Allows field names to be blank (but not null).
Enables schema metadata to be read from JSON integration tests.
Enables integration tests for cases that are now working.

### Are these changes tested?

Yes.

* Closes: apache#36588

Authored-by: Curt Hagenlocher <curt@hagenlocher.org>
Signed-off-by: Curt Hagenlocher <curt@hagenlocher.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@CurtHagenlocher CurtHagenlocher

Labels
Component: C# Component: Python Type: bug
Projects
None yet
Milestone
15.0.0
Development

Successfully merging a pull request may close this issue.

GH-36588: [C#] Support blank column names and enable more integration tests. CurtHagenlocher/arrow
2 participants
@CurtHagenlocher @DanTm99