#22 docs: curate the context surface to the 6,000-token budget (wave A, pair 3)

[gregoryfoster]

Closes #22. Wave A, pair 3 — adopt curating-context v1.2 and run it once.

Before / after

Exact counts via count_tokens (--exact succeeded; tokens_exact: true).

	Before	After
AGENTS.md	19,032	5,773
Lines	309	232
Budget	6,000	6,000 — under
Largest section share	52% (Bus Conventions)	18% (Code Exploration Policy)
Orphaned docs	4	0
Dead links	0	0

The issue quoted 17,521 against remote HEAD; the branch point measured 19,032. Same file, later commits.

tokens_live rose 19,032 → 50,759, which is the expected shape of a successful demotion: the always-paid cost fell 70% while the reachable surface grew because four previously-orphaned docs are now linked and five new ones exist.

Per-section disposition

Section	Before	Class	Disposition	After
(preamble)	66	A	keep	66
Project Overview	354	A	keep	354
Development Methodology	46	A	keep	46
Environment & Tooling	515	A	keep — wheelhouse mechanism + the load-bearing patch floor	543
Code Exploration Policy	1,042	—	untouched; owned by init-socraticode, has its own idempotency contract	1,097
Project Layout	900	B	tree → docs/ARCHITECTURE.md; 6-line signpost stays	138
Infrastructure	533	A+B	single-VM rule + the Redis ≥7.0 floor stay; topology → docs/DEPLOYMENT.md	176
Server Lifecycle	745	A+B	deploy/debug commands + the /etc copy trap stay; table → docs/DEPLOYMENT.md	175
Environment Variables	2,991	A+B	two-env-file boundary stays; per-variable reference → docs/DEPLOYMENT.md	495
Bus Conventions	9,994	A+B	nine condensed rules stay; all reasoning → docs/ARCHITECTURE.md, docs/STORAGE.md, docs/TESTING.md	994
Common Commands	352	A	keep	371
Agent Skills	46	A	keep	46
Conventions	1,494	A+B	commit/date/general rules stay; three logging deep-dives → docs/STYLE.md	541
Detail Docs	—	A	new — the index; class A by construction	493

Bus Conventions at 52% was the finding rather than a symptom, so it was demoted wholesale and rebuilt inline as nine rules that bind every change: two idempotency keys, store-then-publish, count=1, fail-loud from_wire with a global dispatch table, DLQ vs retry, no outbox on the consume path, the three stream kinds, seed_fetch.py as the only writer to content.fetch, and the two normative contracts.

Every relocated block, with its destination

Verbatim moves. prove-no-loss.sh --base <branch-point> accounts for all 226 non-blank lines: 106 still inline, 120 relocated, 0 unaccounted for.

Destination	Tokens	Lines moved	Content
docs/ARCHITECTURE.md	8,091	55	## Project Layout tree (62–92); ## Bus Conventions intro and bullets 189–211 and 214–220
docs/DEPLOYMENT.md	4,232	48	## Infrastructure table + port prose (94–103); ### Redis is Archiver-operated (105–111); ## Server Lifecycle whole (113–136); the environment-variable reference (155–183)
docs/STORAGE.md	1,842	10	blob path and blob-mode bullets (212–213); ### Retention whole (222–231)
docs/STYLE.md	1,034	3	the three logging paragraphs (294, 296, 298)
docs/TESTING.md	905	4	Testing the bus, the fakeredis divergences, real_redis, the dedupe-key namespace (233–239)

Two normalisations the checker permits were the only edits in transit: ### Retention became ## Retention at the top of its section, and ### Redis is Archiver-operated… kept its level under ## Infrastructure. No relative markdown links needed re-levelling — the original file had none (links.refs was empty, which is why every doc was an orphan).

Every destination is under the 10,000 per-doc budget, so the demotion removed cost rather than relocating it.

Deletions

None. Phase 2 returned 41 TRUE, 50 UNVERIFIABLE, 0 FALSE — no warrant existed. The 50 UNVERIFIABLE are the expected classes: absolute VM paths (/etc/replicator/.env), URI schemes (file://, redis://…, gs://…), illustrative templates (src/foo.py), file.py::symbol references, and cross-repo issue numbers. None was treated as a licence to delete.

Also in this PR

• .socraticodecontextartifacts.json re-embedded, not just edited. This repo's own rule is that the manifest is a source and nothing re-embeds it (Consume content.fetch-policy — replace the single env default with Watcher's per-host numbers (cannobserv#285) #19 CR A body-less 304 dead-letters — make "not modified" a first-class outcome before conditional GET is usable #17), so codebase_context_index ran in the same change — 17 artifacts. The five new docs are registered, and agent-guidelines's description now describes a routing file rather than the monolith it used to be.
• Write guard installed (Phase 8). PostToolUse hook, budget 6,000 / doc budget 10,000. Never blocks; silent when an edit reduces the count. The weekly run recovers ground, the guard stops regrowth — a repo with one and not the other sawtooths.
• Telemetry row in .skills/context-metrics.jsonl, --no-loss ok, tagged with nine specific actions.
• docs/SKILLS.md records the v1.2 pin and why breaking it costs the experiment a verdict.

Verification

• uv run pytest — 444 passed, 18 deselected, coverage 99.77%
• uv run ruff check . — clean
• prove-no-loss.sh — exit 0, 0 unaccounted for
• measure-context.sh --exact — links.dead: [], links.orphans: []

Two things this run did not fix

1. docs/contracts/content-fetch-issuer-contract.md is still 14,772 tokens, over the 10,000 per-doc budget. Nothing was demoted into it, per the issue. It needs splitting on its top-level headings — worth its own issue.
2. The daily refresh hook will break the v1.2 pin. .claude/hooks/skills-submodule-update.sh runs git submodule update --remote --merge -- skills-vendor/ once per UTC day on main and commits the result. Left alone it bumps gregoryfoster-skills past 3fc7b71 within a day, which puts two skill versions inside the wave-A arm and makes score-cohort.sh return INCONCLUSIVE. The hook has no opt-out flag and the submodule is one repo, so pinning curating-context alone is not possible — holding the pin means suspending the daily refresh for the duration of the experiment. Flagged rather than decided: that trade is a maintainer call, not a curation one.

🤖 Generated with Claude Code

[gregoryfoster]

Two follow-ups landed on this branch and one filed:

• Daily skills refresh suspended (0f4fded). The hook's SessionStart entry is removed from .claude/settings.json; the script stays, so restoring it is a one-entry edit. docs/SKILLS.md records the pin at 3fc7b71, why the suspension is blunter than the problem, the manual update path while it holds, and when to restore. Upstream remedy tracked at gregoryfoster/skills#100, which is open with no fix yet — this is the same remedy CannObserv/cli took.
• docs: split and compress content-fetch-issuer-contract.md — 14,772 tokens, over the 10,000 per-doc budget #24 filed for splitting and compressing docs/contracts/content-fetch-issuer-contract.md (14,772 tokens, the only live doc still over the 10,000 per-doc budget), with the heading seams, the sibling-link constraint, and the verification command.

[gregoryfoster]

Code review round 1 — 8 findings, all actioned (a60a9b1).

#	Action	Result
1	Fixed	tests/core/test_logging.py — docs/DEPLOYMENT.md is a fourth uvicorn command site since the Server Lifecycle demotion and was outside UVICORN_COMMAND_SOURCES; the --log-config guard was passing by not looking
2	Fixed	README.md pointed at AGENTS.md for "Not everything in the journal is JSON", which moved to docs/STYLE.md on this branch
3	Fixed	Two relocated bullets in docs/DEPLOYMENT.md end see **Testing the bus** / see **Retention** — intra-file at the branch point, cross-file now. Destinations named on continuation lines so the original lines stay verbatim
4	Fixed	docs/ARCHITECTURE.md layout tree read docs/ — Reference docs (COMMANDS, SKILLS)
5	Fixed	Manifest architecture named ./README.md while ./docs/ARCHITECTURE.md was architecture-detail — now readme and architecture, re-embedded
6	Fixed	Dropped the "Replicator is a consumer first" lead-in duplicated between AGENTS.md and docs/ARCHITECTURE.md
7	Documented	docs/SKILLS.md records gregoryfoster/skills#99 — the guard-hook symlink is outside doctor.sh's heal scope and dangles on a submodule-less checkout, failing on every edit. Also completed the manual-update snippet, which omitted the commit the suspended hook used to perform
8	Fixed	README.md's 24-row environment table duplicated docs/DEPLOYMENT.md with neither authoritative — now five rows of what this VM actually overrides, pointing at DEPLOYMENT.md for the rest

One deletion, with its warrant

prove-no-loss.sh --base 6a4b1c0 now reports exactly one line dropped: finding 4's docs/ — Reference docs (COMMANDS, SKILLS).

Warrant: disproven. Refuting command:

$ ls docs/*.md
docs/ARCHITECTURE.md docs/COMMANDS.md docs/DEPLOYMENT.md docs/SKILLS.md docs/STORAGE.md docs/STYLE.md docs/TESTING.md

Seven, not two — and this branch is what made the claim false. The replacement points at the Detail Docs index rather than re-listing, so it cannot go stale the same way again. The --no-loss ok recorded on the ledger row remains accurate for the state it measured; this is a subsequent, reviewed correction.

Gates, re-run after the fixes

• uv run pytest — 444 passed, 18 deselected, coverage 99.77%
• uv run ruff check . / ruff format --check . — clean
• measure-context.sh --exact — policy 5,773, under budget; links.dead: [], links.orphans: []
• Every live doc under the 10,000 per-doc budget except content-fetch-issuer-contract.md (docs: split and compress content-fetch-issuer-contract.md — 14,772 tokens, over the 10,000 per-doc budget #24)
• All local markdown links across AGENTS.md, README.md, and docs/*.md resolve