-
Notifications
You must be signed in to change notification settings - Fork 3.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Go] [Parquet] panic writing with DeltaBinaryPacked when column only has nulls #35718
Comments
Reading more code, it seems possible the fix could be as simple as
(although I could easily believe that the correct response when nil is some constant other than 0) |
@mslevine Thanks for filing this! Would you be interested in filing a PR with a new test case and your proposed fix? 😄 It should automatically tag me as a codeowner to review it if you do. If you're not interested, I'll try to get to this when I have some time. |
…Bytes (#37112) ### Rationale for this change See #37102. Also fixes #35718 Golang BitWriter didn't reserve when `ReserveBytes` called. So it make it wrong. ### What changes are included in this PR? Change `ReserveBytes` impl ### Are these changes tested? Currently not ### Are there any user-facing changes? bugfix * Closes: #37102 Lead-authored-by: Matt Topol <zotthewizard@gmail.com> Co-authored-by: mwish <maplewish117@gmail.com> Signed-off-by: Matt Topol <zotthewizard@gmail.com>
Hi @zeroshade @mapleFU, This issue isn't fixed by #37112. Can you please reopen it? I have tested it locally on the current main and it fails. The proposed fix in this issue seems to work fine as a solution. |
@abhimanyusinghgaur I can reopen this issue, after which can you file a PR with your proposed fix and add a unit test case that can replicate the crash without the fix? |
…eserveBytes (apache#37112) ### Rationale for this change See apache#37102. Also fixes apache#35718 Golang BitWriter didn't reserve when `ReserveBytes` called. So it make it wrong. ### What changes are included in this PR? Change `ReserveBytes` impl ### Are these changes tested? Currently not ### Are there any user-facing changes? bugfix * Closes: apache#37102 Lead-authored-by: Matt Topol <zotthewizard@gmail.com> Co-authored-by: mwish <maplewish117@gmail.com> Signed-off-by: Matt Topol <zotthewizard@gmail.com>
### Rationale for this change closes: #35718 ### What changes are included in this PR? Fix painc writing with DeltaBinaryPacked or DeltaByteArray when column only has nulls ### Are these changes tested? Yes - add a test writing nulls to columns with DeltaBinaryPacked / DeltaByteArray / DeltaLengthByteArray encodings ### Are there any user-facing changes? No * Closes: #35718 Lead-authored-by: yufanmo <yufan.mo@transwarp.io> Co-authored-by: Matt Topol <zotthewizard@gmail.com> Signed-off-by: Matt Topol <zotthewizard@gmail.com>
…e#39497) ### Rationale for this change closes: apache#35718 ### What changes are included in this PR? Fix painc writing with DeltaBinaryPacked or DeltaByteArray when column only has nulls ### Are these changes tested? Yes - add a test writing nulls to columns with DeltaBinaryPacked / DeltaByteArray / DeltaLengthByteArray encodings ### Are there any user-facing changes? No * Closes: apache#35718 Lead-authored-by: yufanmo <yufan.mo@transwarp.io> Co-authored-by: Matt Topol <zotthewizard@gmail.com> Signed-off-by: Matt Topol <zotthewizard@gmail.com>
…e#39497) ### Rationale for this change closes: apache#35718 ### What changes are included in this PR? Fix painc writing with DeltaBinaryPacked or DeltaByteArray when column only has nulls ### Are these changes tested? Yes - add a test writing nulls to columns with DeltaBinaryPacked / DeltaByteArray / DeltaLengthByteArray encodings ### Are there any user-facing changes? No * Closes: apache#35718 Lead-authored-by: yufanmo <yufan.mo@transwarp.io> Co-authored-by: Matt Topol <zotthewizard@gmail.com> Signed-off-by: Matt Topol <zotthewizard@gmail.com>
…e#39497) ### Rationale for this change closes: apache#35718 ### What changes are included in this PR? Fix painc writing with DeltaBinaryPacked or DeltaByteArray when column only has nulls ### Are these changes tested? Yes - add a test writing nulls to columns with DeltaBinaryPacked / DeltaByteArray / DeltaLengthByteArray encodings ### Are there any user-facing changes? No * Closes: apache#35718 Lead-authored-by: yufanmo <yufan.mo@transwarp.io> Co-authored-by: Matt Topol <zotthewizard@gmail.com> Signed-off-by: Matt Topol <zotthewizard@gmail.com>
Describe the bug, including details regarding any error messages, version, and platform.
The writer panics if DeltaBinaryPacked encoding is requested for a column that is all null. The following small change to an existing test case demonstrates the issue.
In case it matters, I'm using
go version go1.20.4 darwin/amd64
I ran into this issue using arrow v12.0.0, but this changed testcase fails using the current head of main.
Component(s)
Go
The text was updated successfully, but these errors were encountered: