-
Notifications
You must be signed in to change notification settings - Fork 3.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
GH-40851: [JS] Fix nullcount and make vectors created from typed arrays not nullable #40852
Conversation
domoritz
commented
Mar 27, 2024
•
edited by github-actions
bot
edited by github-actions
bot
- GitHub Issue: [JS] Typed arrays should not be nullable, null counts should be correct #40851
…d arrays not nullable
@@ -177,16 +180,16 @@ export class Data<T extends DataType = DataType> { | |||
// if we have a nullBitmap, truncate + slice and set it over the pre-filled 1s | |||
if (this.nullCount > 0) { | |||
nullBitmap.set(truncateBitmap(offset, length, this.nullBitmap), 0); | |||
Object.assign(this, { nullBitmap }); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If we assign null count here, the update to the null count will be wrong. Also, we know that the null count has not yet changed so there is no need to set it here.
@@ -177,16 +180,16 @@ export class Data<T extends DataType = DataType> { | |||
// if we have a nullBitmap, truncate + slice and set it over the pre-filled 1s | |||
if (this.nullCount > 0) { | |||
nullBitmap.set(truncateBitmap(offset, length, this.nullBitmap), 0); | |||
Object.assign(this, { nullBitmap }); | |||
} else { | |||
Object.assign(this, { nullBitmap, _nullCount: 0 }); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here we can set the null count to 0 since we just created a new nullBitmap that is all valid values.
@@ -445,7 +445,7 @@ export function makeVector(init: any) { | |||
if (init instanceof DataView) { | |||
init = new Uint8Array(init.buffer); | |||
} | |||
const props = { offset: 0, length: init.length, nullCount: 0, data: init }; | |||
const props = { offset: 0, length: init.length, nullCount: -1, data: init }; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
_nullCount != 0 && (!nullBitmap || nullBitmap.byteLength == 0)
is not nullable so we set -1
as the null count here to mark the data as not nullable. makeData
will always create a null bitmap so I'm not sure why we need to check for no bitmap.
After merging your PR, Conbench analyzed the 7 benchmarking runs that have been run so far on merge-commit 5b09059. There were no benchmark performance regressions. 🎉 The full Conbench report has more details. It also includes information about 14 possible false positives for unstable benchmarks that are known to sometimes produce them. |
…d arrays not nullable (apache#40852) * GitHub Issue: apache#40851
…d arrays not nullable (apache#40852) * GitHub Issue: apache#40851
…d arrays not nullable (apache#40852) * GitHub Issue: apache#40851
…d arrays not nullable (apache#40852) * GitHub Issue: apache#40851
…d arrays not nullable (apache#40852) * GitHub Issue: apache#40851