Skip to content

Say that the benchmark page measures no joins at all - #402

Merged
jdatcmd merged 1 commit into
commandprompt:mainfrom
ChronicallyJD:bench-scope
Aug 4, 2026
Merged

Say that the benchmark page measures no joins at all#402
jdatcmd merged 1 commit into
commandprompt:mainfrom
ChronicallyJD:bench-scope

Conversation

@ChronicallyJD

Copy link
Copy Markdown
Collaborator

docs/benchmarks.md draws conclusions about "analytics". Its evidence covers one shape of
analytics.

The gap

$ grep -lci '\bjoin\b' queries/tsbs/*.sql | wc -l
0
$ grep -hiE '^\s*FROM' queries/tsbs/*.sql | sort | uniq -c
      8 FROM %T

All eight TSBS queries are a scan, a filter and an aggregate over one relation. The page
then closes with:

Columnar is the wrong choice for write-heavy OLTP and the right choice for scan-heavy
and aggregate-heavy analytics over wide, append-mostly tables.

Star schemas and fact-to-dimension joins are a large part of what columnar storage is
bought for. Read as written, that sentence invites a reader to carry a single-table result
into a workload we have never run.

What changed

A What this page does not measure section, placed before the conclusions rather than
after them, saying which shapes are absent and that the absence is unmeasured rather
than bad
. The two conclusions that said "analytics" now say which analytics, and the
closing line notes that whether the result survives a join is open.

Wording is deliberate on one point. It says we have not measured these shapes, not that we
are slow on them. We do not know either way, and the page should not imply confidence in
either direction.

Why now rather than with the measurement

#401 covers building a join-heavy fixture and measuring it. That is real work with a real
answer at the end. The disclosure is true today and costs nothing, and the page is linked
from the 1.0-alpha release notes.

Same class as #381 and #391: the page claiming more than the measurement behind it
supports. This one differs in that nothing measured is wrong, only the scope the
conclusions are drawn over.

Docs only.

The conclusions generalise to "analytics". The evidence does not. All eight TSBS
queries are a scan, a filter and an aggregate over one relation:

  $ grep -lci '\bjoin\b' queries/tsbs/*.sql | wc -l
  0
  $ grep -hiE '^\s*FROM' queries/tsbs/*.sql | sort | uniq -c
        8 FROM %T

Star schemas and dimension joins are a large part of what columnar storage is
bought for, and this page has nothing to say about them. Read as written, it
invited a reader to carry a single-table result into a workload we have never
run.

A new section states the gap before the conclusions rather than after them, and
says plainly that the absence is unmeasured rather than bad. The two conclusions
that said "analytics" now say which analytics, and the closing line notes that
whether the result survives a join is open.

commandprompt#401 covers measuring it. This is the disclosure, which should not wait for the
measurement.

@jdatcmd jdatcmd left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved, and taking a follow-up

The placement is the part that makes this work: before the conclusions, not after.
A caveat under a conclusion is read by nobody who has already read the conclusion.

And the wording restraint is right: "Not that we do badly on them. We have not measured
them." Those are different claims and the page should not imply confidence in either
direction.

One thing it leaves ambiguous, which I can close with evidence

The section says nothing about whether joins work, only that their performance is
unmeasured. A reader can reasonably come away unsure whether a columnar table joins at
all. That is a worse impression than the truth, and I measured the truth earlier today
on a 200,000-row fixture:

shape result
columnar fact JOIN heap dimension, inner + aggregate identical to heap/heap
LEFT JOIN with nulls identical
heap on the outer side identical
columnar JOIN columnar identical
semijoin (EXISTS) / anti-join (NOT EXISTS) identical
hash / merge / nested loop, each forced all correct, all chosen

Column projection also survives the join: the plan reports
Columnar Projected Columns: 1 of 4, so the columnar side reads only the join key
rather than losing the advantage at the join boundary.

So the accurate statement is stronger and more useful: joins are supported and
correct across access methods; their performance is unmeasured.
I will add that in a
follow-up rather than send this back, since the change is additive and you have the
harder half in #401.

The one caveat worth carrying with it: foreign keys are not symmetric across the
boundary. A columnar table cannot be an FK parent, per test/fk_referencing.sh. That
belongs in limitations.md rather than the benchmark page, but it is the thing people
hit immediately after joins.

Merging.

@jdatcmd
jdatcmd merged commit 1509480 into commandprompt:main Aug 4, 2026
11 checks passed
jdatcmd added a commit that referenced this pull request Aug 4, 2026
Joins work; it is their speed that is unmeasured (#402 follow-up)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants