Skip to content

T21: guard §1.7 backward-check against restatement-theatre (cold-sweep agent + enumeration format) - #861

Merged
artyhoo merged 3 commits into
stagingfrom
fix/backward-check-restatement-t21
Jul 2, 2026
Merged

T21: guard §1.7 backward-check against restatement-theatre (cold-sweep agent + enumeration format)#861
artyhoo merged 3 commits into
stagingfrom
fix/backward-check-restatement-t21

Conversation

@artyhoo

@artyhoo artyhoo commented Jul 2, 2026

Copy link
Copy Markdown
Owner

Guard the §1.7 backward-check against restatement-theatre under context fatigue

Codifies a durable countermeasure for a verified incident: deep in a long session, a §1.7 Backward-check degrades from an outward sweep (sibling surfaces where the change-class must also hold) into an inward restatement of the PR's own changes — which passes CI and lets a real parallel gap ship.

The incident (verified, not taken on faith)

PR #857 shipped as two commits on staging history:

  • ec643bac7 — the §1.7 Backward-check restated the PR: «re-adjudicates the S2 §5 risk-acceptances; item 4 promoted documented→coded; items 1-3 kept documented». It named only artifacts the diff already touched.
  • bf1b8b5f3 — after operator challenge, the real sweep found the parallel gap: tier1For's candidateFields (Tier-1 host-derivation) rejected IP-literal / punycode / multi-tenant hosts but not bare TLDs, so homepage:"https://com" derived a trusted com host — the identical single-label hole the PR had just closed on the Tier-2 loadAckFile surface. Fixed with if (!host.includes('.')) continue;.

The theatre backward-check let a real gap reach the PR. The CI gate (discipline-self-check.yml) checks only ≥40 non-whitespace chars + ≥1 file.ext:line citation — both trivially satisfied by a restatement that cites the PR's own diff files.

Why context fatigue produces exactly this (diagnosis)

Deep in a loaded context, working memory is saturated with the PR's inward narrative, so «backward-check» pattern-matches to «recap what I changed» — but the check's entire value is outward (surfaces not in front of you). The check is worst exactly where it is most needed. It composes three catalogued traps: T2 (claiming a sweep vs running it), T14 (clean result on low coverage misreported as clean), T15 (the recursive-self-application the check exists to force is skipped) — reinforced by the fakeable gate that green-lights the least-resistance output (the same evidence-skipping shape as T20 / recommendation-laziness).

Design (evaluated, not defaulted)

  • (a) Cold sub-agent — the structural root fix (foregrounded). agents/backward-sweep-auditor.md: a fresh, PR-blind agent given only the change's class — it enumerates every parallel surface and reports GAP/CLEAN. Immune to restatement by construction: it never saw the PR, so it cannot recap it. Plugs into the in-repo AI-agnostic-auditor pattern (agents/memory-codification-auditor.md) + T19 cold-QA; read in-session, zero paid-LLM. Classified operator-only (skip-loop in install.sh + setup.d/20-agents.sh) like the sibling probers — §1.7 is project-internal.
  • (b) Enumeration authoring format in phase-research-coverage.md §1.7 + the self-reflection skill: Class = X; Surfaces where class-X occurs: [enumerate ALL]; per surface SWEPT-CLEAN | GAP-FOUND. A backward-check whose surface list is exactly the diff's own files is non-conformant by format.
  • (c) Mechanical "cites a non-diff path" detector — REJECTED as a gate, on evidence. The incident's restatement ec643bac7 DID cite a non-diff file (packages/core/principles/30-research-source-trust.test.ts:139, absent from that commit's own git diff --name-only) → a naive non-diff-citation detector false-negatives on the very incident. Completeness of the enumeration is a semantic judgment; per no-paid-llm-in-ci.md it cannot be a CI gate. (Learned from the narrow-b FP-84% lesson — do not ship a noisy/fakeable gate.)

Codified: trap T21 #backward-check-restates-not-sweeps in ai-laziness-traps.md (§2 + §4 anti-pattern + §5 promotion at 3 incidents, MANUAL classification).

§1.7 Forward-check applied

  • no-paid-llm-in-ci.md — the only mechanism is the read-in-session cold agent (agents/backward-sweep-auditor.md:14), zero API-billed calls; T21's §5 promotion note explicitly bars a syntactic CI gate (.claude/rules/ai-laziness-traps.md:178).
  • doc-authority-hierarchy.md §2-§3 — the new agent carries an Authoritative-for header (agents/backward-sweep-auditor.md:11); both edited rule files retain Class + header. Principle 09 green (64/64 across principles 09/12/21/22).
  • build-first-reuse-default.md — ADAPT of the in-repo AI-agnostic-auditor + T19 cold-QA patterns (.claude/rules/ai-laziness-traps.md:144); no new dependency, no capability commit (agent is outside packages/).
  • dual-implementation-discipline.md §2(iv)/§3 — operator-only internal agent → CC-native-only is the correct triage; no portable fallback owed (setup.d/20-agents.sh:28).
  • language-discipline — all new machinery is English (principle 22 green).

§1.7 Backward-check applied

Class of this change = a discipline self-check enforced by a syntactic proxy (section-presence / min-char-length / citation-count) that a shaped-but-empty (or restated-with-self-citation) body can satisfy without doing the substantive work.

Surfaces where the class occurs (population 12, cold-swept via agents/backward-sweep-auditor.md over .github/workflows + .husky + packages/core/hooks/checks + packages/core/principles) — none is in this PR's diff (git diff --name-only origin/staging...HEAD = 7 files, all rules/skill/agent/installer); this is an outward sweep, not a restatement:

Surface Verdict (against EMPTY-filler) Backstop
discipline-self-check.yml:71 §1.7 backward ≥40-char SWEPT-CLEAN citation substance arm :123 + sanity paired-negative :134
s17.ts:94 §1.7: trailer length SWEPT-CLEAN file:line arm s17.ts:96 + body-prose detector
prior-art.ts:160 trailer ≥20 SWEPT-CLEAN placeholder + broken-citation arms
audit-self.yml:499 pr-commit-trailers (CI mirror, PREPUSH_ONLY=s17) SWEPT-CLEAN same TS-core, no re-implemented proxy
13-phase-research-coverage-s17.test.ts:87 hasS17Substance SWEPT-CLEAN bare §1.7 deliberately NOT an arm; C3 paired-negative :226
principles 08 / 15 / 09 / 12 (presence proxies) SWEPT-CLEAN each carries anti-tautology / broken-ref / fenced-example negative

GAP-FOUND: none for the EMPTY-filler sub-class. But the restatement-WITH-self-citation sub-class (T21) is un-gated across all 12 surfaces — a body citing its own diff files clears every length AND citation arm. This is why the fix ships a cold semantic agent, not a 13th syntactic arm.

Recursive self-application, honestly: my hand enumeration named ~4 siblings; the cold sweep enumerated 12. Had I shipped only my hand list, this very backward-check would have been an under-complete sweep — T21 catching its own author. The full self-review (population-12 sweep + forward/backward walk + residual) is hosted discoverably at docs/meta-factory/research-patches/2026-07-03-backward-check-restatement-t21.md, per the §1.7 self-reflexive-trigger convention this change edits (not buried in a commit message).

Honest residual (what it still won't catch)

The countermeasure lowers the cost of doing it right (cold agent) and raises the visibility of doing it wrong (format makes an omission legible at review), but does not make faking impossible. A determined fatigued author can still (i) skip invoking the cold agent and (ii) hand-author a plausible-but-incomplete surface list — no CI gate can verify semantic completeness of an enumeration without a paid LLM (excluded by policy), and the cold sweep confirmed that every syntactic arm is defeated by restatement-with-self-citation. So the trap moves from passive restatement (cheap, default) to active fabrication (requires inventing a false-complete list) — a real raise in cost, backstopped only by the same review-time judgment (human or reviewer-agent) that caught the incident. It is not a gate, by design.

Verification

  • 275/275 principle tests green (packages/core); markdownlint 0 on all edited files.
  • Incident mechanism verified: git show ec643bac7 --name-only | grep -c 30-research-source-trust → 0 (cited-but-not-in-diff, confirming the detector-(c) false-negative).

🤖 Generated with Claude Code

test22345 added 3 commits July 3, 2026 02:03
…eck sweep (T21)

Structural countermeasure for #backward-check-restates-not-sweeps (T21). A cold,
PR-blind sub-agent that receives ONLY a change's *class* (never the diff or PR
narrative) and enumerates every parallel surface in the codebase, reporting
GAP-FOUND / SWEPT-CLEAN per surface. It is PR-blind by dispatch contract: given
only the class, it has no PR narrative in context to recap — the failure mode a
fatigued long-context author falls into when authoring a §1.7 Backward-check.

Classified operator-only (authoring-only) like manual-rule-liveness-prober: added
to the setup.d/20-agents.sh + install.sh --refresh skip-loops (the ship-boundary
SSOT that principle 21-agnosticism reads via parseOperatorOnlyAgents). The agent
body carries the operator-only rationale + a revisit criterion (agents/backward-
sweep-auditor.md): its author-side value is enumerating many parallel sibling
surfaces of framework rule-machinery (a maintainer situation), and a Bash/Grep
sub-agent is broader surface than the shipped read-only reviewer compliance-
verifier.md; reclassify as shipped if consumers report needing it. tools = Read,
Glob, Grep, Bash — all CC-canonical + portable, so it passes both
21-shipped-agent-tools-valid and 21-agnosticism regardless.

Origin incident: PR #857 commit ec643ba shipped a restatement backward-check;
the parallel Tier-1 single-label-host gap it should have swept reached the PR and
was caught only by operator challenge (fixed in bf1b8b5).

§1.7: forward-check — complies with no-paid-llm-in-ci.md (the agent is read by an
active session, zero API-billed calls; agents/backward-sweep-auditor.md:14);
doc-authority-hierarchy.md §2-§3 (the agent carries an Authoritative-for header,
agents/backward-sweep-auditor.md:11; off the principle-09 static list like the
sibling probers — internal authoring tooling, not shipped); dual-implementation-
discipline.md §2(iv)/§3 (operator-only internal agent → CC-native only is the
correct triage, no portable fallback owed); build-first-reuse-default.md (ADAPT of
the in-repo AI-agnostic-auditor pattern — agents/memory-codification-auditor.md:14
— + T19 cold-QA; no new dependency, no BUILD-without-search). backward-check —
Class of change = "an AI-agnostic operator-only sub-agent under agents/". Surfaces
where the class occurs (population 3, via ls agents/ ∩ setup.d/20-agents.sh
skip-loop): agents/manual-rule-liveness-prober.md — SWEPT-CLEAN (header + skip-loop
present, setup.d/20-agents.sh:26); agents/shipped-agent-liveness-prober.md —
SWEPT-CLEAN (setup.d/20-agents.sh:27); agents/backward-sweep-auditor.md — added
here with the same header + skip-loop shape (setup.d/20-agents.sh:28,
install.sh:241). No operator-only agent left without a skip-loop entry — none of
the two pre-existing siblings is in this commit's diff.

Prior-art: skipped — new agents/*.md file (outside packages/, not a capability
commit per CLAUDE.md); ADAPT of the existing AI-agnostic-auditor pattern, no new
dependency or runtime code-module.
…on format

Codifies the countermeasure for the PR #857 backward-check-restatement incident.

- ai-laziness-traps.md §2: new T21 (trigger/tempted/counter/discriminating-tell/
  incident) + §4 anti-pattern #backward-check-restates-not-sweeps + §5 promotion
  (Class A at 3 incidents, MANUAL classification per narrow-b lesson). T21's counter
  is the cold agent + the enumeration format — structural, because a prose reminder
  rots under the exact fatigue that causes the trap.
- phase-research-coverage.md §1.7: the backward-check bullet mandates the enumeration
  authoring format (Class / Surfaces-where-class-occurs / per-surface
  SWEPT-CLEAN|GAP-FOUND) and points to the cold agents/backward-sweep-auditor.md.
- self-reflection SKILL.md + references/backward-checklist.md: Step 0 "defeat
  restatement first".
- docs/meta-factory/research-patches/2026-07-03-backward-check-restatement-t21.md:
  the discoverable self-review (population-12 sweep + forward/backward walk +
  honest residual), per the §1.7 self-reflexive-trigger convention this change edits
  — not buried in a commit message.

Detector (c) ("does the backward-check cite a path not in git diff --name-only") is
REJECTED as a gate, on evidence: PR #857 commit ec643ba's restatement DID cite a
non-diff file (packages/core/principles/30-research-source-trust.test.ts:139, absent
from that commit's own git diff --name-only) → a naive non-diff-citation detector
false-negatives on the very incident. Completeness of the surface enumeration is a
semantic judgment; per no-paid-llm-in-ci.md it cannot be a CI gate → the cold agent
+ review-time judgment carry it. A cold-sweep of this change's own class (12
syntactic-proxy discipline gates) confirmed every gate already backstops the
EMPTY-filler sub-class (Incident-1 stub, 0 citations) but NONE catches
restatement-WITH-self-citation (T21's sub-class) — so a new syntactic arm is not the
fix; a cold semantic sweep is.

§1.7: forward-check — complies with no-paid-llm-in-ci.md (T21 + format are prose; the
only mechanism is the read-in-session cold agent, zero API calls; the §5 T21
promotion bullet explicitly bars a syntactic gate, ai-laziness-traps.md §5);
doc-authority-hierarchy.md §2-§3 (both edited rule files retain Class +
Authoritative-for headers; the research-patch inherits research-patches/README.md
folder authority + carries the principle-10 scope marker; principle 09/10/13 green —
275/275 principle tests pass); build-first-reuse-default.md (REFERENCE the in-repo
AI-agnostic-auditor + T19 cold-QA patterns, ai-laziness-traps.md §2 T19; no new
dependency); recommendation-laziness-discipline.md (T21 is the backward-check
specialisation of T20's evidence-skipping shape). backward-check — Class of this
change = "a discipline self-check enforced by a syntactic proxy (section-presence /
min-char-length / citation-count) that a shaped-but-empty body can satisfy".
Surfaces where the class occurs (population 12, cold-swept via
agents/backward-sweep-auditor.md over .github/workflows + .husky +
packages/core/hooks/checks + principles): discipline-self-check.yml:71 (§1.7 backward
≥40-char) — SWEPT (substance arm :123, sanity paired-negative :134); s17.ts:94
(§1.7: trailer length) — SWEPT (file:line arm s17.ts:96 + body-prose detector);
prior-art.ts:160 (trailer ≥20) — SWEPT (placeholder + broken-citation arms);
audit-self.yml:499 (pr-commit-trailers CI mirror, PREPUSH_ONLY=s17) — SWEPT (same
TS-core, no re-implemented proxy); principle
13-phase-research-coverage-s17.test.ts:87 — SWEPT (bare §1.7 deliberately NOT an
arm, C3 paired-negative :226); principles 08-prior-art-cited.test.ts:90,
15-skill-paired-negative.test.ts:48, 09-doc-authority-hierarchy.test.ts:60,
12-ai-laziness-traps.test.ts:66 — SWEPT (each carries anti-tautology / broken-ref /
fenced-example negative). GAP-FOUND: none for the EMPTY-filler sub-class — but the
restatement-WITH-self-citation sub-class (T21) is un-gated across ALL 12, which is
why this change ships a cold agent, not a 13th syntactic arm. None of the 12
surfaces is in this PR's diff — the sweep is outward, not a restatement of the PR's
own files.

Prior-art: skipped — rule prose + skill edits + a self-review research-patch for an
existing discipline (§1.7); no new dependency or runtime code-module, not a
capability commit.
…ttier gate)

Emphasis markers *italic* → _italic_ to satisfy the shipped-artifacts Prettier
gate (audit-self.yml shipped-prettier, GH #531 dogfood — npm run format:check).
Content unchanged; markdownlint still 0. Formatting-only, no capability.

Prior-art: skipped — prettier emphasis-marker reformat only, no new capability,
dependency, or code-module.
@artyhoo
artyhoo merged commit c07563d into staging Jul 2, 2026
34 checks passed
artyhoo pushed a commit that referenced this pull request Jul 3, 2026
…pe/provenance)

Operationalizes spec §9 from the post-D1 live shapes on feat/diagnostics-core-d1.
Two-gate decomposition (shape FF1xxx / provenance FF2xxx); scopeLock+ackFile fold
into provenance (verified: S2 did NOT ship them separable). Research-side report
mirrors ValidationReport by pattern, not abstraction. tier1ReasonToDiagnostic
cleanup surfaced as DN-B-1 (Option B recommended: trust-tiers handoff, not B).
Pre-merge caveat: authored before D1 merges to staging (#861 F1 owner-blocked);
B's impl gated on D1 landing. All shape claims carry file:line (T3).

Prior-art: skipped — planning doc, no new capability
artyhoo pushed a commit that referenced this pull request Jul 3, 2026
…SSOT #195)

Reconcile the parallel-evolution collision with staging (moved to c07563d during
the D1 run) and unblock the merge, per maintainer 'go' 2026-07-03:

1. Cosmetic conflict in allowlist-resolver.ts — #860 independently added the
   byte-identical bare-TLD Tier-1 guard D1 also has (WF restore cc90b3a + S2-N8);
   resolved keeping D1's Diagnostic|null-migrated version (guard identical).
2. #861 shipped agents/backward-sweep-auditor.md with no Prior-art trailer/SSOT ->
   staging audit-self was RED on principle 11 F1. Closed as an SSOT DATA fix: new
   entry #195 (BUILD verdict, cold PR-blind sibling-surface enumerator, no upstream
   analog) supplies the F1 match. No code change to #861's agent; the underlying gap
   is #861's, fixed here under explicit maintainer authorization to land the chain.

Bundle regenerated (--check green). Full packages/core suite 1893 passed | 12
skipped; principle 11 8/8; typecheck clean post-merge.

Prior-art: skipped — merge reconciliation + an SSOT data-entry (#195) closing an inherited pre-existing F1 gap; no new capability introduced by this merge
artyhoo added a commit that referenced this pull request Jul 3, 2026
…ppend-only FF registry (#862)

Unifies 12 coexisting error models behind one Diagnostic {code, severity, path?, params, message} + an append-only FF registry (packages/core/diagnostics/). Research pipeline gains a zod-style dual API (checkResearchPlan accumulates, validateResearchPlan throws); validateProvenance returns Diagnostic|null; L4 GateFailure gains an additive code field (20 FF3xxx); the two Ajv stacks collapse to one shared factory. Zero behavior change for all current callers; full packages/core suite green (1893 passed).

Also closes an inherited pre-existing staging red: SSOT #195 supplies the Prior-art provenance #861 omitted for agents/backward-sweep-auditor.md (principle 11 F1), under maintainer authorization.

Prior-art: prior-art-evaluations.md#189 (rustc diagnostics, REFERENCE), #190 (tsc diagnosticMessages.json, ADAPT), #191 (SARIF 2.1.0, KEEP NARROW), #192 (zod dual-API, ADOPT VOCABULARY), #193 (Fowler Notification, ADAPT), #194 (ajv shared factory, ADOPT), #195 (backward-sweep-auditor, BUILD).
artyhoo pushed a commit that referenced this pull request Jul 3, 2026
Resolves SSOT conflict: staging's #862 took IDs #189-#195 (D1 diagnostics +
#861 backward-sweep-auditor); this branch's capability-reuse entry renumbered
#189 -> #196 with all in-file references updated (rule, agent, research-patch).
install.sh auto-merged (both SHIPPED_DOCS additions kept). Baselines regenerated
for the current agent content. principle 09/11/21 + byte-identical + format:check
re-verified green post-merge.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants