Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[C++] CSV writer's WriteTable failed with: Failed casting from large_string to string: input array too large #40270

Closed
sighingnow opened this issue Feb 28, 2024 · 1 comment
Closed

[C++] CSV writer's WriteTable failed with: Failed casting from large_string to string: input array too large #40270

sighingnow opened this issue Feb 28, 2024 · 1 comment
Assignees
@sighingnow
Labels
Component: C++ Type: bug
Milestone
17.0.0

Comments

@sighingnow
Copy link
Contributor

Describe the bug, including details regarding any error messages, version, and platform.

For tables which contains arrow::LargeStringArray which further contains long string elements, the following code:

arrow::csv::WriteCSV(*table, options, ofp_.get())

failed with the error: Failed casting from large_string to string: input array too large.

Component(s)

C++
@github-actions github-actions bot mentioned this issue Feb 28, 2024
Merged
pitrou added a commit that referenced this issue Apr 11, 2024
@sighingnow @pitrou
GH-40270: [C++] Use LargeStringArray for casting when writing tables …
7288bd5 
…to CSV (#40271)

### Rationale for this change

Avoid casting failures when tables contains too long large string arrays.

### What changes are included in this PR?

Replace the usage of `StringArray` to `LargeStringArray`.

### Are these changes tested?

No extra test case is needed (as it is to fix some corner cases).

### Are there any user-facing changes?

No user-facing changes.

* GitHub Issue: #40270

Lead-authored-by: Tao He <sighingnow@gmail.com>
Co-authored-by: Antoine Pitrou <antoine@python.org>
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
Signed-off-by: Antoine Pitrou <antoine@python.org>
@pitrou pitrou added this to the 17.0.0 milestone Apr 11, 2024
@pitrou
Copy link
Member

pitrou commented Apr 11, 2024

Issue resolved by pull request 40271
#40271

@pitrou pitrou closed this as completed Apr 11, 2024
vibhatha pushed a commit to vibhatha/arrow that referenced this issue Apr 15, 2024
@sighingnow @pitrou @vibhatha
apacheGH-40270: [C++] Use LargeStringArray for casting when writing t…
e9b3f3c 
…ables to CSV (apache#40271)

### Rationale for this change

Avoid casting failures when tables contains too long large string arrays.

### What changes are included in this PR?

Replace the usage of `StringArray` to `LargeStringArray`.

### Are these changes tested?

No extra test case is needed (as it is to fix some corner cases).

### Are there any user-facing changes?

No user-facing changes.

* GitHub Issue: apache#40270

Lead-authored-by: Tao He <sighingnow@gmail.com>
Co-authored-by: Antoine Pitrou <antoine@python.org>
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
Signed-off-by: Antoine Pitrou <antoine@python.org>
tolleybot pushed a commit to tmct/arrow that referenced this issue May 2, 2024
@sighingnow @pitrou @tolleybot
apacheGH-40270: [C++] Use LargeStringArray for casting when writing t…
10eac5a 
…ables to CSV (apache#40271)

### Rationale for this change

Avoid casting failures when tables contains too long large string arrays.

### What changes are included in this PR?

Replace the usage of `StringArray` to `LargeStringArray`.

### Are these changes tested?

No extra test case is needed (as it is to fix some corner cases).

### Are there any user-facing changes?

No user-facing changes.

* GitHub Issue: apache#40270

Lead-authored-by: Tao He <sighingnow@gmail.com>
Co-authored-by: Antoine Pitrou <antoine@python.org>
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
Signed-off-by: Antoine Pitrou <antoine@python.org>
vibhatha pushed a commit to vibhatha/arrow that referenced this issue May 25, 2024
@sighingnow @pitrou @vibhatha
apacheGH-40270: [C++] Use LargeStringArray for casting when writing t…
69352bd 
…ables to CSV (apache#40271)

### Rationale for this change

Avoid casting failures when tables contains too long large string arrays.

### What changes are included in this PR?

Replace the usage of `StringArray` to `LargeStringArray`.

### Are these changes tested?

No extra test case is needed (as it is to fix some corner cases).

### Are there any user-facing changes?

No user-facing changes.

* GitHub Issue: apache#40270

Lead-authored-by: Tao He <sighingnow@gmail.com>
Co-authored-by: Antoine Pitrou <antoine@python.org>
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
Signed-off-by: Antoine Pitrou <antoine@python.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sighingnow sighingnow

Labels
Component: C++ Type: bug
Projects
None yet
Milestone
17.0.0
Development

No branches or pull requests
2 participants
@pitrou @sighingnow