Source
P0 Task 4 of the guide-behavioral-audit campaign (triage report: docs/superpowers/audits/2026-06-verify-docs-triage.md, run at develop 31b9f0e1f, brew wheels 4.0.3).
What the triage found
Live v4-0-0 tree: 370 tagged blocks → 58 pass / 312 fail — every failure bucket-(c) harness defect, zero verified broken doc claims. The dominant cause: the compile driver's known bracket-balance fallback inverted — wheels CLI 4.0.3's wheels cfml now exits non-zero on errors, so the harness switched into "native" mode, which executes each snippet in a bare Lucee engine with no framework, no app context, no fixtures. Valid Wheels code cannot pass that bar:
- G1 (183 blocks): any
component {...} declaration crashes native eval (Index 0 out of bounds). Representative: testing/index.mdx:40.
- G2 (58): tag-syntax examples get double-wrapped in
<cfscript>.
- G3 (20): describe/it spec fragments need the WheelsTest runner.
- G4 (45): framework config functions (
set(), mapper(), …) / context-dependent fragments need an app.
- Proof of harness-not-content: the harness's own canonical example in its VALIDATION.md fails identically.
Honesty caveat
Because these blocks were never meaningfully compiled (bracket-balance before, context-broken execution now), real content bugs may hide among the 312 — only a fixed harness can find them. That makes this the gating item for the campaign's P0 Task 4 acceptance.
Proposed direction
- A fixture-compile path: wrap snippets by declared kind (component → write to a temp app context and ask the engine to compile/parse, not execute; spec fragment → wrap in a WheelsTest bundle shell; config fragment → include inside a settings fixture). Parse/compile is the bar — execution is a non-goal for
{test:compile}.
- An allowlist/expected-fail mechanism — none exists today (verified: only knobs are target paths,
VERIFY_DOCS_CONCURRENCY, WHEELS_BIN); needed so CI can gate incrementally while groups are fixed.
- Re-run the live tree after (1); only then do real content failures become visible and triagable.
Acceptance
- The G1 representative (
testing/index.mdx:40) and the VALIDATION.md canonical example pass under the new driver.
- Live-tree run reports a meaningful pass rate with failures attributable to content, not context.
docs-verify.yml can gate the live tree without continue-on-error (see the trigger-path fix shipping separately).
Source
P0 Task 4 of the guide-behavioral-audit campaign (triage report:
docs/superpowers/audits/2026-06-verify-docs-triage.md, run at develop31b9f0e1f, brew wheels 4.0.3).What the triage found
Live
v4-0-0tree: 370 tagged blocks → 58 pass / 312 fail — every failure bucket-(c) harness defect, zero verified broken doc claims. The dominant cause: the compile driver's known bracket-balance fallback inverted — wheels CLI 4.0.3'swheels cfmlnow exits non-zero on errors, so the harness switched into "native" mode, which executes each snippet in a bare Lucee engine with no framework, no app context, no fixtures. Valid Wheels code cannot pass that bar:component {...}declaration crashes native eval (Index 0 out of bounds). Representative:testing/index.mdx:40.<cfscript>.set(),mapper(), …) / context-dependent fragments need an app.Honesty caveat
Because these blocks were never meaningfully compiled (bracket-balance before, context-broken execution now), real content bugs may hide among the 312 — only a fixed harness can find them. That makes this the gating item for the campaign's P0 Task 4 acceptance.
Proposed direction
{test:compile}.VERIFY_DOCS_CONCURRENCY,WHEELS_BIN); needed so CI can gate incrementally while groups are fixed.Acceptance
testing/index.mdx:40) and the VALIDATION.md canonical example pass under the new driver.docs-verify.ymlcan gate the live tree withoutcontinue-on-error(see the trigger-path fix shipping separately).