benchmarks: add a columnar batch with long packed columns#318
Conversation
Every packed message in the suite carries a handful of elements per field — PackedTile's widest is 20 — so the benchmarks only ever measure the regime where per-field dispatch dominates. Packed decoding has a second regime, where arrays are long enough that per-element work is the cost, and nothing here reached it. A change targeting that regime could not be shown to work by our own numbers, which is what prompted this. ColumnBatch is a columnar record batch: one field per column, every row's value for that column contiguous on the wire, which is what analytics and telemetry pipelines actually send. 1024 rows per batch, and the columns are picked so each lands in a distinct decode path — verified against the generated bytes rather than assumed: symbol_dict 32 strings, 276 B borrow path, many small spans symbol_ids 1.00 B/elem packed varint, the 1-2 byte inline case timestamps 9.00 B/elem packed varint, the wide case values 8.00 B/elem packed fixed64 weights 4.00 B/elem packed fixed32 Dictionary encoding is how columnar formats keep repeated labels cheap, so the index column earns its place twice: it is realistic, and it is the shape the 1-2 byte varint path is built for. The message is duplicated into proto/iso/ and gated behind its own feature, as the other messages are, so the isolated target compiles no other decoder. The existing isolated benches now name column_batch in their guards too; without that, enabling it alongside one of them would silently defeat the isolation those guards exist to protect. No-Verification-Needed: benchmark-only; column regimes verified against the wire bytes
|
All contributors have signed the CLA ✍️ ✅ |
|
[claude code] The bench earns its keep immediately. Re-ran the #314/#315 metal compare with it included (base = those PRs' merge-base, layout-normalized, spot
−24.66% versus −8.59% from the identical patch: the suite was under-reporting that change roughly threefold, purely because no message here had arrays long enough for per-element work to dominate. That is the gap this PR closes. The column choices pay off in a way worth recording, because it is what makes the two patches separable. Every control is also quiet in this run (−2.13% to +1.73%), against the −12.76% and −26.04% swings |
Adds
ColumnBatch, a columnar record batch with 1024-element packed columns, plus its dataset and benches.Why
Every packed message in the suite carries a handful of elements per field —
PackedTile's widest is 20, its float column is 4-8 and its double column 2-4. Packed decoding has two regimes: short arrays sit under per-field dispatch overhead, long arrays are dominated by per-element work. Nothing in the suite reached the second one, so a change targeting it cannot be shown to work by our own numbers.That is not hypothetical. Reviewing #314 and #315 (bulk packed decode) I could measure their owned-path effect on
packed_tilebut not the effect they are actually for: the contributor's own data puts the win at 64+ elements, and #314's PR body concedes thatPackedTile's 4-8 element arrays "sit under per-field overhead". We were asking a suite to confirm something it structurally cannot see, and the fallback was to trust an out-of-tree workload.The message
A columnar record batch — one field per column, every row's value for that column contiguous on the wire. This is what analytics and telemetry pipelines actually send, and what Arrow and Parquet readers consume, so it earns its place beyond settling one PR.
The columns are picked so each lands in a distinct decode path. Verified against the generated wire bytes rather than assumed:
symbol_dictsymbol_ids(packeduint32)timestamps(packedint64)values(packeddouble)weights(packedfloat)Dictionary encoding is how columnar formats keep repeated labels cheap, so the index column earns its place twice over: it is what a real batch looks like, and it is exactly the shape the 1-2 byte varint path exists for. Timestamps are absolute epoch nanos rather than deltas, which is what makes them wide varints — a delta encoding would have collapsed them into the same regime as the indices and tested one path twice.
~22.9 KB/payload, 1.14 MB dataset — in line with
mesh.pb(875 KB) andmedia_frame.pb(545 KB).Wiring
The message is duplicated into
proto/iso/behind its own feature, as the others are, so the isolated target compiles no other message's decoder. The existing isolated benches now namecolumn_batchin theircompile_error!guards too — without that, enabling it alongside one of them would silently defeat the isolation those guards exist to protect. Verified the guard fires in both directions.Sibling bench crates (prost, prost-bytes, go, google) are untouched: they use only the original five datasets, the same precedent
PackedTile,TriMeshandPackedSignedset.Note for anyone regenerating:
task gen-datasetsrewriteslog_record.pbandmedia_frame.pbwith different bytes every run — those are the two messages withmap<>fields and owned encode iterates aHashMap, so the fixed seed cannot make them reproducible.git checkoutthat drift rather than committing it; onlycolumn_batch.pbis new here.Verification
Decoded the dataset's wire bytes to confirm the per-element widths above; ran the benches (
decode2.26 GiB/s,merge,encode,compute_size,decode_view); built the isolated target with--no-default-features --features iso,column_batch; confirmed both isolation guards fire.cargo fmt --checkandcargo clippy --benchesclean.No changelog fragment: benchmarks ship nothing to users.