docs: re-measure the cross-engine table on current main (#348) - #356
Merged
Conversation
The published cross-engine query latency table was measured on 2026-08-02 at 12:53. Column projection merged at 17:33 the same day, 4.7 hours later. Every figure in it therefore came from a build where the reader loaded and decoded every column of every row group it visited, whatever the query referenced (#338, fixed in #339). It understated the project, which is the direction nobody checks. It published q6 at 81966 ms against TimescaleDB's 1771. Measured today the same shape is 1592 ms against 6956. It also showed pgColumnar losing to heap on q4 and q5, which does not reproduce. This is a new run rather than a row-by-row correction, because the two are not the same experiment. The fixture now carries indexes the earlier one did not, heap moving from 11045 ms to 6 ms on q1, and the earlier queries were never recorded. bench-tsdb/BENCHMARK_RESULTS.md, cited by #289 as the full data and method, has never existed in this repository. So the method is stated here and the SQL is published with the table. TimescaleDB is reported serial and labelled as such. Its parallel path fails on the bench host with "could not read blocks 0..0". That is not caused by pgColumnar: it persists with every pgColumnar planner hook disabled, and the chunks use the heap access method. Neither available workaround is usable, one still errors and the other silently returns zero rows. Citus is omitted rather than carried forward from a run whose configuration is unknown. q1 to q3 are published with their cause attached. They cost about 28.7 ms per row returned, flat across a twelve-fold change in row count, on the same index scan plan heap uses. That is #353, the default stripe_row_limit putting a table this wide over the 32 MB fetch cache, compounded by #355, the planner choosing an index scan for ordering without modelling the fetch cost. Publishing 123 seconds without that would read as a property of columnar storage rather than as one default and one cost model. The storage table is unchanged. Measured 6587 MB, 7975 MB and 22 GB against its published 6.4 GB, 7.8 GB and 22 GB. It was correct. The 2.67 GB figure in #289's body was the wrong one. Results were verified against heap per query. Counts and max match exactly, averages agree to 2.9e-15 relative difference, and no group appears in one engine and not the other. docs_style passes, and the STE line-length rule required splitting several sentences. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011miCFRSatixeNRw3w5yNq8
ChronicallyJD
approved these changes
Aug 3, 2026
ChronicallyJD
left a comment
Collaborator
There was a problem hiding this comment.
Approving — a correct, honest re-measurement and a clear improvement on the stale table.
Verified:
- Method is stated and reproducible, and the SQL is published — the old table's key omission.
- Internal consistency holds: q1/q2 are 28.7 ms/row × 360/4,320 rows; the q6 projection ablation (45,094 → 6,491) matches the #289/#348 numbers; the correctness check (counts and max exact, avg within 2.9e-15) is the expected cross-worker float reassociation.
- q1–q3 are correctly framed as the #353 fetch-cliff defect (compounded by #355), not a storage property, with the smaller-
stripe_row_limitworkaround stated — so the 123 s doesn't misread as columnar being unfit for point queries. - The q5 vectorization regression is disclosed and tracked to #349; the GUC-defaults-off caveat is stated; Citus is honestly omitted rather than carried forward from an unknown config.
- Cited commit
00290d7(#350 merge) is valid; #354 post-dates it but doesn't touch these query shapes.
One follow-up, not a blocker: q1–q3 are exactly the shape my #353 fix (#357) addresses — once #357 lands, those rows should be re-measured (they drop ~200×) and the "until that is fixed" note updated. Fine to merge now with the caveat attached; the re-measure is a small follow-up.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #348.
Why
The published cross-engine table was committed 2026-08-02 12:53. Column projection merged at 17:33 the same day. Every figure in it came from a build where the reader decoded every column of every row group regardless of the query (#338).
It understated the project, which is the direction nobody audits:
Why it is a new run, not a line-by-line correction
Two things make a row-by-row edit misleading:
(hostname, time DESC)btree. Heap goes from 11,045 ms to 6 ms on q1, a factor of ~1,800. That is not drift.bench-tsdb/BENCHMARK_RESULTS.md, cited by Vectorized decompression + aggregation for full-scan aggregates (~4× behind TimescaleDB) #289 as "full data + method", has never existed in this repository.So the method is stated and the SQL is published with the table, which is the main thing the old one lacked.
q1 to q3 ship with their cause attached
They cost ~28.7 ms per row returned, flat across a twelve-fold change in row count, on the same
Index Scanplan heap uses. That is #353 (defaultstripe_row_limitover the 32 MB fetch cache), compounded by #355 (planner takes an index scan for ordering without modelling fetch cost).Publishing q2 at 123 seconds without that would read as a property of columnar storage. It is one default and one cost model, and the doc says so, with the workaround.
TimescaleDB is serial and labelled
Its parallel path fails on the bench host with
could not read blocks 0..0. Not caused by pgColumnar: it persists with every pgColumnar planner hook disabled, and the chunks use the heap AM. Neither workaround is usable, one still errors and the other silently returns zero rows.Citus is omitted rather than carried forward from an unknown configuration.
Storage table unchanged
Measured 6,587 MB / 7,975 MB / 22 GB against its published 6.4 / 7.8 / 22 GB. It was correct. #348 originally suspected it; that suspicion is withdrawn. The 2.67 GB figure in #289's body was the wrong one.
Correctness
Verified per query against heap. Counts and
maxmatch exactly. Averages agree to 2.9e-15 relative difference, the residual being float reassociation across workers. No group appears in one engine and not the other.An earlier md5-based check appeared to show q4, q5 and q6 mismatching. That was the instrument: md5 over the text of
avg()output changes on the last digit. Recorded so it is not rediscovered.Gate
docs_stylepasses. The STE line-length rule required splitting several sentences.🤖 Generated with Claude Code
https://claude.ai/code/session_011miCFRSatixeNRw3w5yNq8