Skip to content

benchmarks: add a columnar batch with long packed columns#318

Merged
iainmcgin merged 1 commit into
mainfrom
iain/bench-column-batch
Jul 17, 2026
Merged

benchmarks: add a columnar batch with long packed columns#318
iainmcgin merged 1 commit into
mainfrom
iain/bench-column-batch

Conversation

@iainmcgin

Copy link
Copy Markdown
Collaborator

Adds ColumnBatch, a columnar record batch with 1024-element packed columns, plus its dataset and benches.

Why

Every packed message in the suite carries a handful of elements per field — PackedTile's widest is 20, its float column is 4-8 and its double column 2-4. Packed decoding has two regimes: short arrays sit under per-field dispatch overhead, long arrays are dominated by per-element work. Nothing in the suite reached the second one, so a change targeting it cannot be shown to work by our own numbers.

That is not hypothetical. Reviewing #314 and #315 (bulk packed decode) I could measure their owned-path effect on packed_tile but not the effect they are actually for: the contributor's own data puts the win at 64+ elements, and #314's PR body concedes that PackedTile's 4-8 element arrays "sit under per-field overhead". We were asking a suite to confirm something it structurally cannot see, and the fallback was to trust an out-of-tree workload.

The message

A columnar record batch — one field per column, every row's value for that column contiguous on the wire. This is what analytics and telemetry pipelines actually send, and what Arrow and Parquet readers consume, so it earns its place beyond settling one PR.

The columns are picked so each lands in a distinct decode path. Verified against the generated wire bytes rather than assumed:

column bytes/element path
symbol_dict 32 strings, 276 B total borrow path, many small spans
symbol_ids (packed uint32) 1.00 packed varint, the 1-2 byte inline case
timestamps (packed int64) 9.00 packed varint, the wide case
values (packed double) 8.00 packed fixed64
weights (packed float) 4.00 packed fixed32

Dictionary encoding is how columnar formats keep repeated labels cheap, so the index column earns its place twice over: it is what a real batch looks like, and it is exactly the shape the 1-2 byte varint path exists for. Timestamps are absolute epoch nanos rather than deltas, which is what makes them wide varints — a delta encoding would have collapsed them into the same regime as the indices and tested one path twice.

~22.9 KB/payload, 1.14 MB dataset — in line with mesh.pb (875 KB) and media_frame.pb (545 KB).

Wiring

The message is duplicated into proto/iso/ behind its own feature, as the others are, so the isolated target compiles no other message's decoder. The existing isolated benches now name column_batch in their compile_error! guards too — without that, enabling it alongside one of them would silently defeat the isolation those guards exist to protect. Verified the guard fires in both directions.

Sibling bench crates (prost, prost-bytes, go, google) are untouched: they use only the original five datasets, the same precedent PackedTile, TriMesh and PackedSigned set.

Note for anyone regenerating: task gen-datasets rewrites log_record.pb and media_frame.pb with different bytes every run — those are the two messages with map<> fields and owned encode iterates a HashMap, so the fixed seed cannot make them reproducible. git checkout that drift rather than committing it; only column_batch.pb is new here.

Verification

Decoded the dataset's wire bytes to confirm the per-element widths above; ran the benches (decode 2.26 GiB/s, merge, encode, compute_size, decode_view); built the isolated target with --no-default-features --features iso,column_batch; confirmed both isolation guards fire. cargo fmt --check and cargo clippy --benches clean.

No changelog fragment: benchmarks ship nothing to users.

Every packed message in the suite carries a handful of elements per field —
PackedTile's widest is 20 — so the benchmarks only ever measure the regime
where per-field dispatch dominates. Packed decoding has a second regime, where
arrays are long enough that per-element work is the cost, and nothing here
reached it. A change targeting that regime could not be shown to work by our
own numbers, which is what prompted this.

ColumnBatch is a columnar record batch: one field per column, every row's value
for that column contiguous on the wire, which is what analytics and telemetry
pipelines actually send. 1024 rows per batch, and the columns are picked so each
lands in a distinct decode path — verified against the generated bytes rather
than assumed:

  symbol_dict  32 strings, 276 B      borrow path, many small spans
  symbol_ids   1.00 B/elem            packed varint, the 1-2 byte inline case
  timestamps   9.00 B/elem            packed varint, the wide case
  values       8.00 B/elem            packed fixed64
  weights      4.00 B/elem            packed fixed32

Dictionary encoding is how columnar formats keep repeated labels cheap, so the
index column earns its place twice: it is realistic, and it is the shape the
1-2 byte varint path is built for.

The message is duplicated into proto/iso/ and gated behind its own feature, as
the other messages are, so the isolated target compiles no other decoder. The
existing isolated benches now name column_batch in their guards too; without
that, enabling it alongside one of them would silently defeat the isolation
those guards exist to protect.

No-Verification-Needed: benchmark-only; column regimes verified against the wire bytes
@github-actions

Copy link
Copy Markdown

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@iainmcgin

Copy link
Copy Markdown
Collaborator Author

[claude code] The bench earns its keep immediately. Re-ran the #314/#315 metal compare with it included (base = those PRs' merge-base, layout-normalized, spot c7i.metal-24xl):

benchmark #314 #315 (cumulative)
column_batch/decode (1024-elem columns) −24.66% −36.23%
column_batch/decode_view −15.57% −20.48%
packed_tile/decode (4-20 elem) −8.59% −16.33%
packed_signed/decode (varint-only) −0.06% −7.62%
mesh/decode (control) −2.13% +1.73%
mesh/decode_view (control) −3.00% −3.61%

−24.66% versus −8.59% from the identical patch: the suite was under-reporting that change roughly threefold, purely because no message here had arrays long enough for per-element work to dominate. That is the gap this PR closes.

The column choices pay off in a way worth recording, because it is what makes the two patches separable. packed_signed is varint-only, so under #314 (fixed-width only) it reads −0.06% — the exact zero it should — and #315 moves it to −7.62%. column_batch carries both regimes at once (~12 KB fixed-width, ~10 KB varint of a 22.9 KB payload), so it measures each patch alone and both together. A single mixed column, or delta-encoded timestamps, would have blurred that.

Every control is also quiet in this run (−2.13% to +1.73%), against the −12.76% and −26.04% swings mesh/decode_view produced on the pre-#318 build pairs. I don't claim the new bench caused that — layout noise is per-build-pair luck — but it does mean the view-path numbers above are usable, which none of my earlier runs' were.

@iainmcgin
iainmcgin marked this pull request as ready for review July 16, 2026 22:29
@iainmcgin
iainmcgin enabled auto-merge July 16, 2026 22:32
@iainmcgin
iainmcgin added this pull request to the merge queue Jul 16, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jul 16, 2026
@iainmcgin
iainmcgin added this pull request to the merge queue Jul 16, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jul 16, 2026
@iainmcgin
iainmcgin added this pull request to the merge queue Jul 17, 2026
Merged via the queue into main with commit 029d1a4 Jul 17, 2026
9 checks passed
@iainmcgin
iainmcgin deleted the iain/bench-column-batch branch July 17, 2026 00:19
@github-actions github-actions Bot locked and limited conversation to collaborators Jul 17, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants