Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

GH-35948: [Go] Only cast int8 and unit8 to float64 when JSON marshaling arrays #35950

Merged
merged 3 commits into from
Jun 6, 2023
Merged

GH-35948: [Go] Only cast int8 and unit8 to float64 when JSON marshaling arrays #35950

merged 3 commits into from
Jun 6, 2023

Conversation

erezrokah
Copy link
Contributor

@erezrokah erezrokah commented Jun 6, 2023 •
edited by github-actions bot
Loading

Rationale for this change

See issue.

Based on

arrow/go/arrow/array/numeric.gen.go.tmpl

Line 120 in 9fb8697

{{else if (eq .Size "1")}}
it seems we should only do the casting for unit8 and int8 types.

What changes are included in this PR?

This PR fixes the numeric template so it only casts int8 and unit8 to float64 when JSON marshaling arrays to avoid losing data for other types.

Are these changes tested?

Yes

Are there any user-facing changes?

The user facing change is that if a user was JSON marshaling arrays that has uint64 or int64 values they can now get different (more accurate) values than before

@erezrokah erezrokah requested a review from zeroshade as a code owner June 6, 2023 15:56
@zeroshade zeroshade changed the title GH-35948: [GO] Only cast int8 and unit8 to float64 when JSON marshaling arrays GH-35948: [Go] Only cast int8 and unit8 to float64 when JSON marshaling arrays Jun 6, 2023
@erezrokah
Copy link
Contributor Author

Not sure if the failing test is related to flaky, help appreciated

@zeroshade
Copy link
Member

@erezrokah The failing test is indeed a flaky test, I haven't yet been able to reproduce it locally in order to fix it which annoys me. But you're fine.

zeroshade
zeroshade approved these changes Jun 6, 2023
View reviewed changes
Copy link
Member

@zeroshade zeroshade left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks much for this!

@zeroshade zeroshade merged commit 745fa94 into apache:main Jun 6, 2023
@github-actions github-actions bot added awaiting merge Awaiting merge and removed awaiting review Awaiting review labels Jun 6, 2023
@erezrokah
Copy link
Contributor Author

Thanks for the quick review and merge!

@erezrokah erezrokah deleted the fix/array_dont_cast_int64_unit64 branch June 6, 2023 16:39
@ursabot
Copy link

ursabot commented Jun 7, 2023

Benchmark runs are scheduled for baseline = 105b9df and contender = 745fa94. 745fa94 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Finished ⬇️2.04% ⬆️0.03%] test-mac-arm
[Finished ⬇️0.0% ⬆️0.0%] ursa-i9-9960x
[Failed ⬇️0.15% ⬆️0.06%] ursa-thinkcentre-m75q
Buildkite builds:
[Finished] 745fa94f ec2-t3-xlarge-us-east-2
[Finished] 745fa94f test-mac-arm
[Finished] 745fa94f ursa-i9-9960x
[Failed] 745fa94f ursa-thinkcentre-m75q
[Finished] 105b9df0 ec2-t3-xlarge-us-east-2
[Finished] 105b9df0 test-mac-arm
[Finished] 105b9df0 ursa-i9-9960x
[Failed] 105b9df0 ursa-thinkcentre-m75q
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

zeroshade pushed a commit to zeroshade/arrow that referenced this pull request Apr 11, 2024
@erezrokah @zeroshade
apacheGH-35948: [Go] Only cast int8 and unit8 to float64 when J…
8ace49b 
…SON marshaling arrays (apache#35950)

### Rationale for this change

See [issue](apache#35948).

Based on https://github.com/apache/arrow/blob/9fb8697dcb442f63317c7d6046393fb74842e0ae/go/arrow/array/numeric.gen.go.tmpl#L120 it seems we should only do the casting for `unit8` and `int8` types.

### What changes are included in this PR?

This PR fixes the numeric template so it only casts `int8` and `unit8` to `float64` when JSON marshaling arrays to avoid losing data for other types.

### Are these changes tested?

Yes

### Are there any user-facing changes?

The user facing change is that if a user was JSON marshaling arrays that has `uint64` or `int64` values they can now get different (more accurate) values than before

* Closes: apache#35948

Authored-by: erezrokah <erezrokah@users.noreply.github.com>
Signed-off-by: Matt Topol <zotthewizard@gmail.com>
(cherry picked from commit 745fa94)
@zeroshade zeroshade mentioned this pull request Apr 11, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@zeroshade zeroshade zeroshade approved these changes

Assignees
No one assigned
Labels
awaiting merge Awaiting merge Component: Go
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[Go] Marshaling int64 or unit64 arrays to JSON can cause loss of data
3 participants
@erezrokah @zeroshade @ursabot