Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GH-39257: [JS] LargeBinary #39258

Merged
merged 4 commits into from
Dec 18, 2023
Merged

GH-39257: [JS] LargeBinary #39258

merged 4 commits into from
Dec 18, 2023

Conversation

domoritz
Copy link
Member

@domoritz domoritz commented Dec 17, 2023

Comment on lines +39 to +54
protected _flushPending(pending: Map<number, Uint8Array | undefined>, pendingLength: number) {
const offsets = this._offsets;
const data = this._values.reserve(pendingLength).buffer;
let offset = 0;
for (const [index, value] of pending) {
if (value === undefined) {
offsets.set(index, BigInt(0));
} else {
const length = value.length;
data.set(value, offset);
offsets.set(index, BigInt(length));
offset += length;
}
}
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you want to activate the placeholder in builder/largeutf8.ts in this PR?

Seems like you could probably do the same for getByteLength, too.

(LargeUtf8Builder.prototype as any)._flushPending = (LargeBinaryBuilder.prototype as any)._flushPending;

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, just updated the pull request.

Copy link
Contributor

@bmschmidt bmschmidt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, fwiw, except for the duplicated code between the utf8 and binary builders. The new unification stuff is nice.

@domoritz
Copy link
Member Author

Oh good catch. I forgot to remove that duplicate code.

@domoritz
Copy link
Member Author

CC @kylebarron

Copy link
Contributor

@kylebarron kylebarron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@github-actions github-actions bot added awaiting changes Awaiting changes awaiting change review Awaiting change review and removed awaiting committer review Awaiting committer review awaiting changes Awaiting changes labels Dec 18, 2023
@domoritz domoritz merged commit 4ec6544 into apache:main Dec 18, 2023
10 checks passed
@domoritz domoritz deleted the dom/largeBinary-2 branch December 18, 2023 03:27
Copy link

After merging your PR, Conbench analyzed the 6 benchmarking runs that have been run so far on merge-commit 4ec6544.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 7 possible false positives for unstable benchmarks that are known to sometimes produce them.

clayburn pushed a commit to clayburn/arrow that referenced this pull request Jan 23, 2024
dgreiss pushed a commit to dgreiss/arrow that referenced this pull request Feb 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[JS] Support LargeBinary type
3 participants