Skip to content

docs: the ClickBench shape labels describe the queries they are attached to (#533) - #535

Merged
ChronicallyJD merged 2 commits into
mainfrom
docs/533-clickbench-shape-labels
Aug 9, 2026
Merged

docs: the ClickBench shape labels describe the queries they are attached to (#533)#535
ChronicallyJD merged 2 commits into
mainfrom
docs/533-clickbench-shape-labels

Conversation

@ChronicallyJD

Copy link
Copy Markdown
Collaborator

Closes #533. (Supersedes #534, which GitHub auto-closed when its base branch
was deleted on merge of #526. Same branch, same commit, now based on main.)
Six of the ten query shape labels in the 2026-08-05 ClickBench section described
a different query. The timings are fine. Only the shape column is wrong, which is
the awkward part: the numbers look checkable and the prose beside them was not
inviting a check.

Losses table

doc said the query actually is
q24: SearchPhrase LIKE, ORDER BY EventTime LIMIT 10 SELECT * FROM hits WHERE URL LIKE '%google%' ORDER BY EventTime LIMIT 10
q23: SELECT * of every column a GROUP BY SearchPhrase with Title LIKE '%Google%' and COUNT(DISTINCT UserID)
q21 correct
q28: GROUP BY a normalised URL GROUP BY CounterID with AVG(length(URL)). The normalised URL is q29
q22: SearchPhrase LIKE, ORDER BY EventTime LIMIT 10 GROUP BY SearchPhrase with URL LIKE '%google%', ordered by the count

The SELECT * label sat on q23 when it belongs to q24, and the paragraph under
the table inherited it word for word.

Wins table

doc said the query actually is
q1, q3, q7 correct
q41, q42: GROUP BY a URL prefix q41 groups by URLHash, EventDate, q42 by WindowClientWidth, WindowClientHeight
q20: COUNT(*) with a LIKE on a short column SELECT UserID FROM hits WHERE UserID = 435090932899640449, a point lookup with no COUNT and no LIKE

It is not an off-by-one

That was the first hypothesis, and it is a good one: upstream ClickBench numbers
from q0 and the harness prints from q1. Two labels fit the shift exactly, the
doc's q23 being the file's q24 and the doc's q28 being the file's q29. But q3, q7
and q21 fit the 1-based numbering and not the shifted one, and "GROUP BY a URL
prefix" fits neither q41/q42 nor q42/q43. So it is not a systematic shift.

The magnitudes settle which numbering the table uses. The doc's largest loss is
q24 at 11.6x, and the file's q24 is the SELECT * query, which is also the
largest loss on the 2026-08-09 run at 6.19x. The doc's q41 and q42 are 25x wins,
and those are 17.8x and 16.8x wins on 2026-08-09. The numbering is 1-based and
only the prose was wrong: the labels were written from memory rather than read
off the definition.

What this does not claim

The 2026-08-05 run predates the definition digest being recorded, so there is no
proof it fetched this queries.sql. The section now says the labels are matched
against the definition as recorded on 2026-08-06,
queries.sql a7d6673357348ee9, rather than implying the run itself was pinned.
Claiming more would be claiming something that cannot be checked.

test/docs_style.sh passes 6/6.

jdatcmd and others added 2 commits August 9, 2026 10:49
…hed to (#533)

Six of the ten query shape labels in the 2026-08-05 section described a
different query. The timings were fine; only the prose beside them was wrong,
which is the awkward shape of it, because the numbers look checkable and the
labels did not invite checking.

The worst two: the losses table put "SELECT * of every column" on q23 when it
belongs to q24, and the paragraph under the table inherited it. And q28 was
labelled "GROUP BY a normalised URL", which is q29. In the wins table, q20 was
called "COUNT(*) with a LIKE on a short column" when it is a point lookup,
SELECT UserID WHERE UserID = <constant>, with no COUNT and no LIKE.

It is not an off-by-one, which was the first hypothesis, because upstream
ClickBench numbers from q0 while the harness prints from q1. Two labels fit the
shift exactly and three fit the 1-based numbering instead, and one fits neither.
The magnitudes confirm the numbering: the doc's largest loss is q24 at 11.6x and
the file's q24 is the SELECT * query, which is also the largest loss on the
2026-08-09 run. So the labels were written from memory rather than read off the
definition.

The shapes are now read off the definition as recorded on 2026-08-06,
queries.sql a7d6673357348ee9. That run predates the digest being recorded, so
the section now says the labels are matched against the recorded definition
rather than proved against the file that run fetched. Claiming more would be
claiming what cannot be checked.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WmQJqcXdwyuoAiHHt2znBr
@ChronicallyJD
ChronicallyJD merged commit d39017c into main Aug 9, 2026
11 checks passed
@ChronicallyJD
ChronicallyJD deleted the docs/533-clickbench-shape-labels branch August 9, 2026 17:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

docs: six of the ten ClickBench query shape labels in the 2026-08-05 section describe a different query

2 participants