Skip to content

Phase 1: the exit-code contract and machine-readable output (ADR-0007) - #24

Merged
JumpTechCode merged 1 commit into
mainfrom
phase1/machine-contracts
Jul 26, 2026
Merged

Phase 1: the exit-code contract and machine-readable output (ADR-0007)#24
JumpTechCode merged 1 commit into
mainfrom
phase1/machine-contracts

Conversation

@JumpTechCode

Copy link
Copy Markdown
Collaborator

Third of three Phase 1 records. Two independent adversarial reviews found three
things worth catching before anything was published.

The verdict exit codes were wrong, in the one direction that matters

The design justified 70–73 as "clear of sysexits' common values". Verified
against sysexits.h on this machine:

EX_SOFTWARE   70   internal software error
EX_OSERR      71   system error
EX_OSFILE     72   critical OS file missing
EX_CANTCREAT  73   can't create output file

Not near sysexits — four consecutive values inside its 64–78 block. And the
collision at 70 is semantically inverted: the convention reserves it for an
internal error in the program itself, which is what Assayer means by 82, not 80.
A wrapper dying of its own fault at 70 would have been reported as a regression
that never happened — the manufactured verdict this design forbids everywhere
else, arriving through the exit code rather than through any verdict logic.

Moved to 80–83, above the sysexits block and below the shell's 126/127.
Nothing had been published, so this was free today and a breaking change to
strangers' scripts after the first release.

Precedence was unspecified

A suite can carry a failing exam, a rotted pin, and an errored one at once — the
decision rule makes that routine. With no stated order, two conforming
implementations disagree, and the worst case is a real regression resolving to
the stale-pin code and being waved through. Now specified and tested:
82 > 80 > 81 > 83 > 0, with 82 first because it says the other answers could
not be trusted.

A shipped bug: trailing arguments were ignored

$ assayer version --jsonn
v0.0.0-...          # exit 0

A script asking for a machine-readable document got prose and a success code.
Flags are now checked against what each command accepts; that case exits 2.

Machine-readable output starts

assayer version --json emits the first document. Each carries its schema
identifier, a revision that moves whenever an emitted shape changes, the
stability tier, and the list of schemas the build can produce — discovery
without running anything that costs money.

Schemas are generated from the types, not hand-maintained beside them, and
asserted against the committed files. A hand-written schema plus an external
validator is two descriptions that can be wrong together: rename a field,
regenerate the sample, and it still validates. This is the arrangement the repo
already uses for its linter rules, and it needs no dependency — go.mod stays
empty.

Verified by planting a field: both the schema check and the golden fail.

assayer.dev is someone else's product

The draft rooted schema identifiers there, on the stated assumption that nothing
was served at that address. Not checked, and false — it is registered and
serving "Assayer – Production Readiness Review for Vibe-Coded Apps". Identifiers
are now rooted at this repository. The namesake collision is worth a separate
look at ADR-0002; this PR only stops the contract depending on it.

Stated rather than implied

The event stream and verdict objects are not built. They land before Phase 1
closes, with the components that produce them — and no command returns a verdict
code until a machine-readable verdict object ships alongside it, because an alarm
with no readout leaves scraping human output as the only option, and those
scrapers become an unversioned contract.

The exam format's schema is also deferred, carrying a real question: the format
is TOML, the stdlib cannot parse TOML, so validating it needs either this
module's first dependency or a different format. Recorded so it stops being
invisible.

make verify green: coverage 90.2%, 0 lint issues, 0 vulnerabilities.

Three findings, all verified rather than argued.

The verdict range moves from 70-73 to 80-83. The design justified 70-73 as
"clear of sysexits' common values"; sysexits.h defines EX_SOFTWARE 70,
EX_OSERR 71, EX_OSFILE 72 and EX_CANTCREAT 73, four consecutive values inside
its 64-78 block. The collision at 70 is the damaging one: the convention
reserves it for an internal error in the program itself, which is what this
program means by 82. A wrapper dying of its own fault at 70 would have been
reported as a regression that never happened — the manufactured verdict
forbidden everywhere else, arriving through the exit code rather than through
verdict logic. Nothing had been published, so the correction was free.

Precedence is now specified: 82, then 80, then 81, then 83, then 0.
Simultaneity is routine — a suite can carry a failing exam, a rotted pin and an
errored one at once — and without an order two conforming implementations
disagree, with a real regression resolving to the stale code and being waved
through as the worst case.

Trailing arguments were silently ignored, so `assayer version --jsonn` printed
human text and exited 0: a script asking for a machine-readable document got
prose and a success code. Flags are now checked against what each command
accepts.

Machine-readable output starts with `version --json`. Documents carry the
schema identifier, a revision that moves whenever an emitted shape changes,
the stability tier, and the list of schemas the build can produce, so a
consumer can discover what it is talking to without running anything that
costs money. Schemas are generated from the types that produce the documents
and asserted against the committed files, rather than hand-maintained beside
them and checked by a validator this module would have needed its first
dependency for.

Identifiers are rooted at this repository. An earlier draft used assayer.dev on
the assumption that nothing was served there; that was not checked and is
false — the domain is registered and serving an unrelated product of the same
name.

schemas/ joins the architecture guard's scan roots: a committed schema
enumerating adapter names would otherwise name harnesses in a public artifact
that nothing was looking at.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ew64uotJqtvVDN3twMAVEQ
@codecov

codecov Bot commented Jul 26, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 94.00000% with 3 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
internal/cli/cli.go 86.36% 2 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

@JumpTechCode
JumpTechCode merged commit 23aa265 into main Jul 26, 2026
6 checks passed
@JumpTechCode
JumpTechCode deleted the phase1/machine-contracts branch July 26, 2026 22:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant