perf(rowids): accelerate dense bitmap decoding - #8715
Merged
Xuanwo merged 6 commits intoSep 2, 2026
Conversation
This was referenced Aug 24, 2026
jiaoew1991
force-pushed
the
perf/rowid-bitmap-fast-paths
branch
from
August 24, 2026 12:08
2e7f052 to
95b4c9c
Compare
jiaoew1991
changed the base branch from
jiaoew/stack-rowid-cursor-base
to
jiaoew/stack-rowid-cursor-base-v2
August 24, 2026 12:14
jiaoew1991
force-pushed
the
perf/rowid-bitmap-fast-paths
branch
from
August 24, 2026 13:53
95b4c9c to
c69253b
Compare
jiaoew1991
changed the base branch from
jiaoew/stack-rowid-cursor-base-v2
to
jiaoew/stack-rowid-cursor-base-v3
August 24, 2026 13:54
jiaoew1991
force-pushed
the
perf/rowid-bitmap-fast-paths
branch
from
August 24, 2026 14:33
c69253b to
4e6c51e
Compare
jiaoew1991
changed the base branch from
jiaoew/stack-rowid-cursor-base-v3
to
jiaoew/stack-rowid-cursor-base-v4
August 24, 2026 14:33
jiaoew1991
force-pushed
the
perf/rowid-bitmap-fast-paths
branch
from
August 25, 2026 01:09
4e6c51e to
470d561
Compare
jiaoew1991
force-pushed
the
perf/rowid-bitmap-fast-paths
branch
from
September 2, 2026 09:26
19f4049 to
e60b7b3
Compare
Contributor
There was a problem hiding this comment.
✅ Gate recommendation: approve.
The latest merge from main leaves the reviewed bitmap patch unchanged relative to its new base, and the incoming changes do not touch the row-ID path. The focused crate suite still passes, so the previously verified dense-decoding speedup, sparse fallback, cursor behavior, public bitmap mutation, and byte-exact serialization remain sound.
Xuanwo
approved these changes
Sep 2, 2026
Xuanwo
added a commit
that referenced
this pull request
Sep 3, 2026
## Summary Stream dataset-version RLE metadata with a persistent cursor instead of rebuilding and searching run offsets for every batch. The change: - adds `RowDatasetVersionCursor` for adjacent, monotonic batch traversal; - caches the active run length and bulk-expands range values; - lazily builds a run-offset index on the first backward seek, then uses it for every later non-adjacent seek in either direction; - precomputes created-at and last-updated-at arrays while polling the ordered task stream; - preserves the direct-call fallback and the single-run fast path. #8713 and #8715 are merged. This branch is rebased on `774e32d67`, and its diff now contains only the dataset-version work. ## Performance Linux Criterion A/B, 100,000 rows, batch size 1,024, `release-with-debug`, pinned to the same CPU. Baseline is `774e32d67`; this PR is `6f22e566c`. ### Ordered stream | Version runs | main | This PR | Result | |---:|---:|---:|---:| | 1 | 98.325 us | 98.855 us | +0.54% (neutral) | | 98 | 930.69 us | 112.30 us | 8.29x faster | | 3,125 | 3.2053 ms | 128.27 us | 24.99x faster | | 100,000 | 56.033 ms | 517.17 us | 108.34x faster | ### Direct-call fallback | Version runs | main | This PR | Result | |---:|---:|---:|---:| | 1 | 631.81 ns | 634.07 ns | +0.36% (neutral) | | 98 | 8.983 us | 762.10 ns | 11.79x faster | | 3,125 | 32.532 us | 5.108 us | 6.37x faster | | 100,000 | 552.08 us | 142.84 us | 3.87x faster | This PR makes no single-run speedup claim. It improves robustness for updated and multi-run fragments. For attribution, Linux `perf` sampled the 3,125-run ordered-stream case for five seconds at 997 Hz. On main, 49.72% of samples were in `version_values_for_selection`, 39.39% in `U64Segment::len`, and 4.96% in run-offset `Vec` construction (94.07% combined). With this PR, `U64Segment::len` and offset construction were each below the 0.5% report threshold; the remaining profile was distributed across cursor expansion, allocation, Arrow column assembly, and stream scheduling. ## Correctness The tests cover adjacent ranges, gaps, rewinds, empty runs, non-range spans, out-of-bounds selections, descending seeks, alternating far-forward/backward seeks, direct calls, deletions, both version columns, concurrent batches, and unsorted indices split across multiple batches. ## Validation - `cargo test -p lance-table --lib` (372 passed) - `cargo clippy --all --tests --benches -- -D warnings` - `cargo fmt --all -- --check` - `git diff --check` - Linux `release-with-debug` Criterion A/B and `perf` profiles described above --------- Co-authored-by: Xuanwo <github@xuanwo.io>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add dense and near-dense bitmap decode paths.
This follows #8713 and targets
RangeWithBitmapsegments after the sequential cursor has removed repeated prefix scans.The decoder now:
0xffbyte as one contiguous eight-value range;The sparse cursor and bitmap loop remain separate and source-equivalent to
main. This avoids the measurable fallback regression caused by performing adaptive dispatch in every batch.Bitmap.dataandBitmap.lenremain publicly accessible for source compatibility. A proposed popcount cache was removed because direct mutation of the public byte vector could otherwise make the cached cardinality stale. The on-disk encoding remains byte-for-byte unchanged.Performance
Measured on Linux x86_64 with
release-with-debug, 1,000,000 output rows, batch size 1,024, 10 Criterion samples, 1 second warm-up, and 3 seconds measurement. Both binaries used the same benchmark source. Baseline was currentmainatd57d0fb42; candidate wasfbde7d600.mainholes_2)holes_2)holes_17)holes_17)Linux
perfattributes the dense-shape improvement to the intended decoder change: onmain,SegmentCursorState::extend_rangeaccounts for 68.17% of CPU samples; this PR moves that work toSegmentCursorState::extend_dense_range(48.61% of samples) while reducing end-to-end time by 35.58%. For the 50%-density fallback, bothmainand this PR remain inSegmentCursorState::extend_range(66.83% and 70.13% respectively); no adaptive-dispatch helper appears in the hot path.Validation
cargo test -p lance-table --lib(365 passed)cargo clippy --all --tests --benches -- -D warningscargo fmt --all -- --checkgit diff --checkU64Segment::RangeWithBitmapDependency #8713 is merged. This PR targets
maindirectly and contains only the bitmap follow-up.