perf(arrow/array): bulk append fixed-width empty values - #1180
Open
fallintoplace wants to merge 1 commit into
Open
perf(arrow/array): bulk append fixed-width empty values#1180fallintoplace wants to merge 1 commit into
fallintoplace wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
AppendEmptyValuesused to callAppendEmptyValuein a loop. For larger batches, each value repeated the reserve, bitmap, and storage work.This change makes fixed-width empty appends bulk operations:
The numeric generated source and template are updated together. The covered builders are numeric, temporal, interval, float16, decimal, fixed-size binary, and boolean.
Correctness
bufferBuilder.Advancenow zeroes skipped bytes as documented.BooleanBuilder.Resizenow preserves the requested logical length when truncating.Benchmark
AppendEmptyValues(1024)on an Apple M1 Pro. Three runs were used for each result.Small counts keep the single-value path, so the biggest improvement is for larger batches.
Tests
go test ./arrow/arraygo vet ./arrow/array