Skip to content

Analytical docs in STE, and a gate that makes the rule durable (#291) - #298

Merged
jdatcmd merged 2 commits into
mainfrom
docs/291-ste-analytical
Jul 31, 2026
Merged

Analytical docs in STE, and a gate that makes the rule durable (#291)#298
jdatcmd merged 2 commits into
mainfrom
docs/291-ste-analytical

Conversation

@jdatcmd

@jdatcmd jdatcmd commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator

Completes the documentation half of #291, and then does the part that makes it
last: it checks the rule.

Five-major matrix: ALL VERSIONS PASSED, docs_style green on all five.

Converted

file long sentences before after
limitations.md 51 0
benchmarks.md 22 0
testing.md 16 0
ARCHITECTURE.md 15 0
features.md 17 0
index.md 1 0

features.md and index.md were in neither scope list. A project-wide rule has
to cover them, so they are in.

Structural rules only for these, as agreed: they argue rather than instruct, and
the restricted vocabulary would blunt them. Two data-flow paragraphs in
ARCHITECTURE.md became numbered steps instead. They were diagrams written as
prose, and splitting them into sentences would have kept the length rule and lost
the shape.

The durable part

test/docs_style.sh runs in the matrix. A document that drifts goes red.

A rule that nothing checks is a rule the next writer does not know about, and this
project has paid for that shape once already: an empty REGRESS made
make installcheck report success while running nothing.

Four rules over docs/*.md and README.md: no em or en dash, no double hyphen
used as a dash in prose, 25 words to a sentence, no phrase from an idiom list.

Scope is a decision, not an oversight, and the suite says so:

  • CHANGELOG.md is a record of what happened when it happened. Rewriting landed
    entries would edit history, so it is checked for dash characters only.
  • design/ and the code comments are out of scope. Both explain why a thing is
    the way it is, and that reasoning is worth more than uniformity.

Proved by removal, each rule separately

33-word sentence added   -> FAIL ... 1 long
"falls back to" added    -> FAIL ... 1 idiom
em dash added            -> FAIL ... 1 em/en dash
em dash in CHANGELOG     -> FAIL CHANGELOG.md carries no em or en dash

Plus an anti-vacuity control: the suite asserts it examined at least ten
documents, so a broken glob cannot pass as a clean run.

My first attempt at the length proof used a 24-word sentence and did not fire.
That was my error and not the checker's, and it is the reason the proof above
states the word count.

What is not claimed

Full ASD-STE100 compliance. It is defined against the licensed ASD Dictionary
of roughly 900 approved words, which this project does not have. The
approved-vocabulary rule is not enforced and is not claimed, in the suite, in the
checker, or in docs/testing.md. An unverifiable claim of compliance would be
worse than an honest partial one.

One thing worth reporting

The gate found three violations in the section of docs/testing.md that
documents the gate
. That is the most direct evidence I can offer that it works.

jdatcmd and others added 2 commits July 31, 2026 10:49
…291)

The instructional documents were converted in #296. This converts the analytical
documents, converts the two that were in neither list, and then does the part that
makes the rule survive: it checks it.

A rule that nothing checks is a rule the next writer does not know about. This
project has already paid for that shape once, when an empty REGRESS made
`make installcheck` report success while running nothing. So `test/docs_style.sh`
runs in the matrix and a document that drifts goes red.

Converted here: limitations, benchmarks, testing, ARCHITECTURE, features, index,
and one sentence in README. The structural rules only, since these documents argue
rather than instruct, and the restricted vocabulary would blunt them.

  file             long sentences before   after
  limitations.md                      51       0
  benchmarks.md                       22       0
  testing.md                          16       0
  ARCHITECTURE.md                     15       0
  features.md                         17       0
  index.md                             1       0

Two data-flow paragraphs in ARCHITECTURE.md became numbered steps rather than
sentences. They were diagrams written as prose, and splitting them into sentences
would have kept the length rule and lost the shape.

The gate enforces four rules over docs/*.md and README.md: no em or en dash, no
double hyphen used as a dash in prose, 25 words to a sentence, and no phrase from
an idiom list. CHANGELOG.md is a record of what happened when it happened, so
rewriting landed entries would edit history; it is checked for dash characters
only. design/ and the code comments are out of scope on purpose: both explain why
a thing is the way it is, and that reasoning is worth more than uniformity.

What is NOT claimed, in the suite, in the checker and in the documentation: full
ASD-STE100 compliance. That is defined against the licensed ASD Dictionary of
approximately 900 approved words, which this project does not have. The
approved-vocabulary rule is not enforced and is not claimed. An unverifiable
claim of compliance would be worse than an honest partial one.

Each of the four rules is proved by removal, not assumed:

  33-word sentence added   -> FAIL ... 1 long
  "falls back to" added    -> FAIL ... 1 idiom
  em dash added            -> FAIL ... 1 em/en dash
  em dash in CHANGELOG     -> FAIL CHANGELOG.md carries no em or en dash

The suite also carries an anti-vacuity control: it asserts it examined at least
ten documents, so a broken glob cannot pass as a clean run.

The gate found three violations in the section of docs/testing.md that documents
the gate, which is the most direct evidence that it works.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@jdatcmd
jdatcmd merged commit 88a167b into main Jul 31, 2026
10 checks passed
ChronicallyJD pushed a commit that referenced this pull request Aug 1, 2026
The refresh in #320 rebuilt the open list from issue STATE. The follow-up commit on
this branch fixed the #155 entry but repeated the same mistake on the entry it
wrote to replace it. An audit of every entry against its issue thread, its pull
requests and main found that all four were wrong, in three different ways.

#289 was a copy of the issue body and gave no sign that work is in flight. The
decompression half already merged (#307, 3.8 percent on q4 and 3.2 on q5) and the
aggregation half is open as #321. The "about 4x behind TimescaleDB" line reads as
the size of the prize for that work, but #321 measures 1.20x and 1.38x, and by its
own account the larger lever is dictionary-coded grouping. The widest gap, q6 at
5.3x behind and 3.1x slower than heap, is the only shape where columnar loses to
heap and nothing in flight touches it.

#300 was framed as core COPY's per-field parse. #300's own profile refuted that
before the entry was written: parse is about 21 percent, encode about 53 percent,
so bypassing the parser cannot make columnar beat heap. The measured top lever is
parallelism over the existing encoder with COPY unchanged, prototyped at 7.39x.
IMPORT_THROUGHPUT_PLAN.md was cited as the reference and is the wrong pointer: it
predates the #283 to #286 work and puts COPY under "Not in scope".

reltuples is removed. It was fixed on 2026-07-28 by #189 and is now exact on every
measured shape, and the cause the entry gave was explicitly disproven: it was a
block-offset mismatch, not blocks holding no row-group data. The line was written
about nine hours before the fix and survived two refreshes.

#310 is no longer listed as work. Both causes are merged and it was re-measured at
100M, 273,212 buffers to 8,917. It stays open for a confirmation reading on the
real dataset.

#291 was open and absent from the list; added, with the note that its
documentation half landed in #298.

Also fixed, all verified: the "Deferred, not yet built" paragraph listed two things
that have been on main since 2026-07-23; a cross-reference to "item 0" that #320's
renumbering left dangling; six Done rows naming the extension schema as columnar
rather than pgcolumnar, which a reader copying them would find does not exist; and
a closed-since line with the wrong date and three omissions.

Refs #289, #300, #291, #310. No issue is closed by this commit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011miCFRSatixeNRw3w5yNq8
ChronicallyJD pushed a commit that referenced this pull request Aug 2, 2026
The refresh in #320 rebuilt the open list from issue STATE. The follow-up commit on
this branch fixed the #155 entry but repeated the same mistake on the entry it
wrote to replace it. An audit of every entry against its issue thread, its pull
requests and main found that all four were wrong, in three different ways.

#289 was a copy of the issue body and gave no sign that work is in flight. The
decompression half already merged (#307, 3.8 percent on q4 and 3.2 on q5) and the
aggregation half is open as #321. The "about 4x behind TimescaleDB" line reads as
the size of the prize for that work, but #321 measures 1.20x and 1.38x, and by its
own account the larger lever is dictionary-coded grouping. The widest gap, q6 at
5.3x behind and 3.1x slower than heap, is the only shape where columnar loses to
heap and nothing in flight touches it.

#300 was framed as core COPY's per-field parse. #300's own profile refuted that
before the entry was written: parse is about 21 percent, encode about 53 percent,
so bypassing the parser cannot make columnar beat heap. The measured top lever is
parallelism over the existing encoder with COPY unchanged, prototyped at 7.39x.
IMPORT_THROUGHPUT_PLAN.md was cited as the reference and is the wrong pointer: it
predates the #283 to #286 work and puts COPY under "Not in scope".

reltuples is removed. It was fixed on 2026-07-28 by #189 and is now exact on every
measured shape, and the cause the entry gave was explicitly disproven: it was a
block-offset mismatch, not blocks holding no row-group data. The line was written
about nine hours before the fix and survived two refreshes.

#310 is no longer listed as work. Both causes are merged and it was re-measured at
100M, 273,212 buffers to 8,917. It stays open for a confirmation reading on the
real dataset.

#291 was open and absent from the list; added, with the note that its
documentation half landed in #298.

Also fixed, all verified: the "Deferred, not yet built" paragraph listed two things
that have been on main since 2026-07-23; a cross-reference to "item 0" that #320's
renumbering left dangling; six Done rows naming the extension schema as columnar
rather than pgcolumnar, which a reader copying them would find does not exist; and
a closed-since line with the wrong date and three omissions.

Refs #289, #300, #291, #310. No issue is closed by this commit.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011miCFRSatixeNRw3w5yNq8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant