📝 docs: AUD-S06 truth-lag sweep — front-of-house surfaces describe the shipped product (DOC-05/06/07/09/10/11) - #21
Merged
Conversation
…k-start (DOC-07) `assent lint .assent/` exits 2: discoverAssentTree joins `.assent` onto the path it is given (`cmd/assent/lint.go`), so the documented command looked for `.assent/.assent`. Every reader who followed the README quick-start verbatim since it was written hit that. Same for `assent test .assent/`. The prose now says why the argument is the repository, not the policy directory, and names a shipped sample tree for readers without a repo of their own. `hack/docs/readme_smoke_test.sh` stops this recurring by construction: it extracts the fenced bash blocks under "## Quick start" and RUNS each `assent` line against a freshly built binary in a throwaway copy of the sample repo the README itself names. `go install` and `task` lines are skipped with a printed reason and a counter, and a block with zero executed commands fails — so the gate cannot go vacuous. Discrimination proven, not assumed: with `assent lint .assent/` restored in a scratch copy of the tree, the script exits 1 naming the failing command. REQ-AUD-S06-01.
…d version (DOC-11)
The README opened the quick-start with "Install a stamped binary" directly above a
`go install` block, and install.md offered `@v0.1.0` as the reproducible-toolchain
route. Neither is true: only goreleaser passes `-X main.version`, so `go install`
leaves the compile-time default in place regardless of the ref.
Verified by execution, not by reading the ldflags:
$ GOBIN=... go install github.com/PlatformRelay/assent/cmd/assent@v0.1.0
$ assent version
assent 0.0.0-dev
Both surfaces now say so and route readers who need a true version string at the
Homebrew tap or a release archive. The install page's "Once tagged releases publish
(E9-S05/S06)" preamble also goes — they publish; v0.1.0 is out.
REQ-AUD-S06-02 (DOC-11).
`docs/adr/0014-policy-test-harness.md` has never existed; the adopter test format ADR is `docs/adr/0014-adopter-test-format.md`. `mkdocs --strict` cannot see this — README.md is outside `docs_dir` — so the link 404'd on GitHub unnoticed. REQ-AUD-S06-02 (DOC-05).
…mplemented" note (DOC-06)
E-FILEEVENTS landed whole-file lifecycle matching, but the stability contract still
told adopters the domain did not exist, and two doc comments in the policy model said
the same. The note now states what actually holds and where the boundary is:
- `internal/change.FileEvent` mints whole-file add/delete changes;
- `LoadMergePolicy` accepts `match.fileEvents.kinds` ⊆ {add, delete} and REJECTS
modify/rename at load with a located error;
- the frozen schema still accepts the full four-kind enum, so widening later is
additive and needs no apiVersion bump.
Verified by running the binary, not by reading the loader. `kinds: [add, delete]` in
`examples/packs/service-catalog` lints clean and its delete case decides BLOCK;
adding `rename` to that same rule turns `assent lint` red with
match.fileEvents kind "rename" is not supported — only add and delete whole-file
events are emitted; modify and rename are deferred
`docs/api-stability.md` is the published mirror of the root file and is regenerated
from it (only the link prefixes differ). The policy.go edit is comment-only —
`git diff` contains no non-comment line.
REQ-AUD-S06-02 (DOC-06).
…ign-fiction header (DOC-09)
The published walkthrough opened with "This is design fiction. Nothing below is
implemented." That was false in both directions after v0.1.0: run/lint/test/doctor/
compare ship, while init/scan/stats/explain do not exist at all. A reader who believed
the banner skipped a working product; a reader who ignored it typed commands that
are not in the dispatch table.
Every step now carries a Shipped or Planned banner matching the dispatch table the CLI
reference is pinned to, each Planned step names what to do today instead, and a status
summary table closes the page. Three specific fictions replaced with verified truth:
- Step 1's `assent init --sample topic-registry` -> copy `examples/packs/
topic-registry/.assent`, which I ran: `assent lint` on the copy is clean.
- Step 3's invented `assent test` output -> the real output of
`assent test examples/packs/topic-registry`, pasted from the run; flags and exit
codes taken from the pinned CLI reference.
- Step 5's `image: ghcr.io/<org>/assent:v0` -> an in-job install. No container image
is published: `.goreleaser.yaml` declares no `dockers:` block and no workflow
pushes to ghcr.io.
Also corrected: the observe-phase name is `observe`, not `advise` (ADR-0018 §1 enum
`off | observe | enforce`).
REQ-AUD-S06-02 (DOC-09).
…executed (DOC-10) The public meta-plan still carried the Phase-2 *proposed* cut — E2 "Decision engine + Rego frontend", E3 "Declarative YAML frontend", E8 "Forge: GitHub adapter" — which contradicts the README maturity table (Rego = E11, GitHub = E10) and every spec directory under `openspec/specs/`. A reader comparing the two pages could not tell which numbering was live. Rows are now derived from the spec titles that actually exist, each row names its spec directory, and the deferred tiers (E10-E13) are listed separately with the pointer to the README table. The three off-sequence Phase-5 epics (EFE, PCS, AUD) are named too. REQ-AUD-S06-02 (DOC-10).
The ADR that specified the checked-file completeness contract still read `Status: Proposed` while the mechanism it specifies is in `main`: `forge.Snapshot` carries `ChangedFilesComplete`/`ChangedFilesGap`, the GitLab adapter paginates `/diffs` with the `changes_count` cross-check and the page ceiling, an unprovable enumeration folds to an opaque changeset and lands on the frozen `changeset.undecidable` axis, and the point-6 conformance obligations have catalog rows tagged `adr: ADR-0020` with tests behind them. Every other implemented ADR in the index reads `Accepted (…)`. Status and the index row now say so, using the sibling convention of naming the decision that accepted it (D-119). No qualifier is needed — points 1-6 all landed; this is not a partial acceptance. REQ-AUD-S06-02 (routed truth-lag item).
…harness caveat
Two stale claims on a shipped surface. The banner said "Pre-alpha: schemas are
illustrative drafts ... the authoritative contracts are frozen in meta-plan Phase 3" —
both halves are false: Phase 3 froze (ADR-0018/0019 read `Accepted (D-030 —
Phase-3 freeze review)`) and the authored surfaces are the frozen
`assent.dev/v1alpha1` schemas under `schemas/`. Removed rather than reworded; no part
of it survives. The intro's "must pass the adopter test harness (`assent test`) once
it exists" goes the same way — it exists, and the packs pass under it.
The index also listed 3 of the 9 example directories. All nine are listed now, each
with what actually holds for it, verified by running the gate that covers it:
- `examples/packs/**` lint clean and test green — pinned by
TestExamplesPacksLoadAndLintClean and TestAllExamplePacksGreenUnderAssentTest,
both inside `task check`; confirmed by hand for all three packs.
- `examples/lint-fixtures/**` good/bad polarity — pinned by
TestEveryHardErrorFixtureCaught.
- `examples/policies/rego/` is marked illustrative: the Rego backend is deferred
tier E11, so that file is a sketch, not a runnable path.
REQ-AUD-S06-02 (routed truth-lag item).
… -checkout flag AUD-S01 changed what a checkout-less run does when the forge cannot prove it enumerated every changed file: the change set becomes opaque and the decision degrades to REVIEW with `changeset.undecidable`. The CLI reference still described `-checkout` purely as "when unset, only the governed subject is diffed", which reads as a scoping choice rather than a fail-safe with an observable outcome — an operator debugging a surprise REVIEW had nothing here to explain it. Adds a short section under `assent run` covering the completeness proof, the REVIEW degradation and its finding code, the still-BLOCK case for a visible `.assent/**` path, and the D-077 checkout-mode carve-out. The fenced help block is untouched — it stays pinned byte-for-byte to the binary (AUD-S05); this is prose and one flag-table cell only. REQ-AUD-S06-02 (routed truth-lag item).
…SEC-07) The v0.1.0 assets were replaced in place twice after publication. Each replacement was re-signed and re-attested consistently, so the audit accepted the outcome — but a consumer who had already verified the previous checksum has no signal that the bytes changed. One runbook line so the next occurrence is a patch tag instead, and so `workflow_dispatch` on an existing tag stays scoped to recovering a failed upload. No process machinery beyond this: SEC-07 is an accepted finding, and the epic's non-goals rule out anything heavier. REQ-AUD-S06-02 (SEC-07 runbook line).
…ck (DOC-05/06/09/10/11)
`hack/docs/truthlag_pins_test.sh` guards the surfaces AUD-S05's TestNoStaleProductClaims
does not walk — repo-root markdown and examples/ — plus four drift pairs no build step
checks:
DOC-05 every in-repo relative link in README.md resolves on the filesystem.
`mkdocs --strict` cannot see these: README.md is outside docs_dir, which is
exactly why the dead ADR-0014 link survived.
DOC-06 neither API-stability copy claims something is "not yet implemented", both
still carry the fileEvents note (so the first pin cannot go vacuous), and
docs/api-stability.md stays byte-identical to the root file modulo the two
docs-relative link prefixes.
DOC-09 no design-fiction banner, and EACH `## Step` heading is followed by its own
Shipped/Planned banner before the next heading.
DOC-10 the Phase-5 table has all nine E1..E9 rows and binds no deferred tier
(Rego/GitHub/serve) to one of those numbers.
DOC-11 README and install.md name the version `go install` actually prints.
plus docs/adr/README.md's status column agrees with each ADR's own Status row —
the drift that left ADR-0020 "Proposed" in two places at once.
Every pin was proven to discriminate, not assumed to: nine mutations applied one at a
time to a scratch copy of the tree (banner restored, link broken, mirror drifted, epic
renumbered, caveat deleted, ADR status reverted, ...) and each turned the script red
naming its finding. The first draft of the DOC-09 check compared banner COUNTS and
survived deleting a step's banner; that is why it is now a per-step walk. The
walkthrough header drops the words "design fiction" so the phrase pin stays exact.
REQ-AUD-S06-02.
…ng (D-120) D-120 landed on main while this lane was open: `pins.toolDigest` now derives from the binary's Go build info, so a `go install` build IS distinguishable in a DecisionRecord. The DOC-11 caveat written earlier in this lane overshot — it said such a binary "cannot identify itself in a DecisionRecord", which D-120 made false. What actually holds is narrower: `pins.toolVersion` reads `0.0.0-dev` and cannot be mapped back to a released tag. The walkthrough's CI job used `go install` with no note, which contradicted the install page's own advice on the one path that emits records. It now states the tradeoff inline and points at the archive route. REQ-AUD-S06-02 (DOC-11).
`readme_smoke_test.sh` told the reader "`task check` IS the gate this script runs under", and printed the same claim at runtime when it skipped a `task` line. Neither script is invoked by `Taskfile.yml` or any workflow — they run only by hand. A tool built to kill unverified claims was carrying one about itself. Both now state the truth: the wiring is intended, is Lane B's to add, and is recorded as D-124; the skip reason says the recursion would happen *once wired*. Both headers carry an explicit NOT YET WIRED note with the consequence. REQ-AUD-S06-01, REQ-AUD-S06-02.
… Lane B owns the wiring The two truth-lag gates from this lane are green and mutation-proven but nothing invokes them, and that fact existed nowhere in the repository — only in a session transcript, which is to say nowhere. D-124 records it durably: the scripts, why AUD-S06 could not wire them (`Taskfile.yml` and `.github/workflows/**` are Lane B's), the assignment (a `docs-gates` task as a `check:` dependency, following the `hack/compare/exitgate_test.sh` precedent from D-118), and the consequence in plain words — until then a README edit reopens DOC-07 with nothing going red. The row also carries the two known pin gaps to extend when wiring: the DOC-09 check asserts banner presence, not polarity; the DOC-05 link loop covers README.md only, so the links added to examples/README.md are unpinned. REQ-AUD-S06-02.
… exist All three `examples/packs/*/.assent/config.yaml` opened with "(assent init --sample <x>)". These trees are meant to be COPIED into an adopter's repository — the walkthrough now tells readers to do exactly that, because there is no `assent init` in the dispatch table — so the false claim was shipping verbatim into every adopter repo that followed the documented path, in the one file they are most likely to read first. Each header now says what to do instead and points at the walkthrough. All three packs still lint clean. REQ-AUD-S06-02 (routed truth-lag item).
All three were live only in a session transcript or in gitignored agent-context, which
is the same "a follow-up that lives nowhere does not exist" gap D-124 opened for.
Recorded, not fixed — each is editorial and widening the reviewed diff costs more than
it buys.
Verified before writing, not restated:
- meta-plan.md's "Ordering constraint: E7 starts early (alongside E1)" now sits
directly under a heading asserting the table is the numbering that executed. It
isn't: E6's tip ec91226 is an ancestor of E7-S08 f27457d, so E7 landed after E6.
- install.md credits a stamped version to "the Homebrew bottle". `.goreleaser.yaml`'s
`brews:` block publishes a Formula whose `url_template` points at the release
archive; no bottle is built or hosted. The version claim is true, the term is not.
- AUD-S05's TestNoStaleProductClaims walks `..`, `../../internal`, `../../docs` — so
hack/, .github/ and test/ markdown are covered by no pin, and that file is not
AUD-S06's to widen.
D-124.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes audit findings DOC-05, DOC-06, DOC-07, DOC-09, DOC-10, DOC-11, the stale
internal/core/policy/policy.gocomments, and the SEC-07 runbook line.Why
v0.1.0 is released and Homebrew-installable, but the surfaces a prospective adopter touches first
still described the pre-release state: a quick-start whose first command exits 2, a walkthrough
carrying a "design fiction" banner with invented output and a
ghcr.io/<org>/assent:v0image thatdoes not exist, an
API_STABILITY.mdclaimingfileEventsis unimplemented, and an epic tablenaming the wrong technologies.
Verified by execution, not by reading
Every correction was checked by running the thing with the released v0.1.0 binary:
assent lint .assent/genuinely exits 2; the argument is the repo root.go install …/cmd/assent@v0.1.0produces a binary reportingassent 0.0.0-dev,while both README and
install.mdclaimed that path gives a stamped, reproducible toolchain. Theconsequence is narrowed correctly post-D-120:
pins.toolDigestdoes identify such a binary; onlypins.toolVersionis untrue.assent testoutput, and thecontainer reference is gone (
.goreleaser.yamlhas nodockers:block and no workflow pushes toghcr).
match.fileEventsis implemented and narrowed to{add, delete}at the loader whilethe schema keeps the four-kind enum, so widening stays additive.
Proposed→Acceptednow that AUD-S01 shipped its mechanism.Kept true by two executable gates
hack/docs/readme_smoke_test.shextracts the fenced quick-start blocks and runs eachassentline;
hack/docs/truthlag_pins_test.shcarries 18 pins. Review planted 20 mutations — all 20 wentred — and confirmed both scripts under GNU grep + mawk in Docker as well as macOS bash 3.2, with no
dead patterns, no vacuous negatives and no SIGPIPE trap.
Two findings worth naming
task checkIS the gate this scriptruns under" — which is exactly the defect class this story closes, written into the tool built to
close it. Reworded to assert intent and ownership; both scripts now carry an explicit
NOT-YET-WIRED header.
assent init --samplein a config header, and the walkthrough tells adopters to copy that same.assent/tree into their own repo. Corrected in all three.D-124 records the wiring gap durably in-repo — assigning a
docs-gatestask to the lane thatowns
Taskfile.yml, and folding in three further known-unfixed residuals, each verified before beingwritten down.
Refs: DOC-05/06/07/09/10/11, SEC-07, D-124, ADR-0020 · REQ-AUD-S06-01/02