GH-32947: [Go] Implement Concatenate for REE Array #14126
Conversation
zeroshade
commented
Sep 14, 2022
•
zeroshade
commented
- Closes: [Go] Concatenate Implementation for REE Array #32947
|
CC @zagto |
zeroshade
force-pushed
the
rle-concatenate
branch
2 times, most recently
from
fb90e87
to
aa29f37
Compare
zeroshade
force-pushed
the
rle-concatenate
branch
from
bccb88f
to
230ff03
Compare
zeroshade
changed the title
|
|
go/arrow/array/concat.go
Outdated
|
|
||
| physicalLength, overflow = addOvf(physicalLength, int(plen)) | ||
| if overflow { | ||
| return nil, fmt.Errorf("%w: run length encoded array length must fit into a 32-bit signed integer", |
There was a problem hiding this comment.
Hmm, shouldn't this depend on the offset type?
There was a problem hiding this comment.
I suppose there are separate checks below, and since lengths are limited to 32-bit...
There was a problem hiding this comment.
so, this is more of a short circuit. We have to add up all of the physical lengths so we know the size to allocate for the full concatenated array. If we end up overflowing when adding up the physical length we don't have to bother calculating the new run ends, we know we're going to overflow no matter what the offset type is. So this doesn't need to depend on the offset type.
There was a problem hiding this comment.
Fair enough. I suppose you could short circuit here earlier for int16, but as long as it properly fails either way.
zeroshade
changed the title
|
I'll merge this at EOD if no one else has comments |
|
Benchmark runs are scheduled for baseline = 4a1229c and contender = 3b21e0c. 3b21e0c is a master commit associated with this PR. Results will be available as each benchmark for each run completes. |
* Closes: apache#32947 Authored-by: Matt Topol <zotthewizard@gmail.com> Signed-off-by: Matt Topol <zotthewizard@gmail.com>
* Closes: apache#32947 Authored-by: Matt Topol <zotthewizard@gmail.com> Signed-off-by: Matt Topol <zotthewizard@gmail.com>
