Skip to content

v0.3.0

Choose a tag to compare

@pedromvgomes pedromvgomes released this 06 Sep 23:12
· 13 commits to main since this release
430b2d2

Drive Codex for non-interactive single-turn runs, and bind a run's final answer to a JSON Schema.

Codex, and a stream-first interface

The codex provider is written against captured output from the real CLI, and what it found reshaped the interface. codex exec --json is JSONL from its first line to its last and has no envelope to parse — -o/--output-last-message writes bare text to a file rather than a document to stdout — so a Result exists only as a fold over a whole run. Streaming is therefore mandatory on Provider, and Run is Stream folded to its terminal Result.

A terminal result outranks both the exit code and the timeout. Codex reports a rejected credential and an unsupported model by finishing its stream properly and then exiting non-zero; judging the exit code first turns those verdicts into spurious outages.

A turn limit is a capability, not a field every provider honours. Codex has no turn bound at any spelling — -c max_turns=N is accepted and silently ignored without --strict-config — and no per-tool allowlist of any kind, so it refuses MaxTurns and AllowedTools rather than accepting and discarding them.

Schema-constrained structured output

Request.Schema binds a run's final answer to a JSON Schema and Result.Structured carries it, so a caller can fan out N reviewer runs over a diff and unmarshal what comes back.

Both CLIs genuinely constrain rather than suggest — given a schema requiring count between 1000 and 2000 and a prompt insisting on 4, each returns a conforming document. They constrain by different mechanisms that fail differently: codex constrains the decoder, so an unsatisfiable schema is a grammar it cannot finish and the run burns to its output ceiling and reports turn.failed; Claude Code validates each StructuredOutput call, feeds the rejection back, and when the model gives up answers in prose on exit 0 with is_error: false and subtype: "success".

A run that was given a schema and produced no payload is reported as an unmet constraint — IsError set, Structured nil, Text carrying whatever account exists. It is the one verdict the library reaches on its own, and it outranks a sandbox refusal on the same run: refusing is a successful verdict about authority, but the caller still did not get the shape it asked for.

The schema reaches each CLI differently — inline for Claude Code, a file path for codex. The codex file is content-addressed so one request always renders one argv, lives in a per-account directory that is checked before use, and is read back and compared rather than trusted for having the right name.

Breaking changes

For anyone implementing Provider:

  • Command and Parse are replaced by StreamCommand and NewDecoder(Request); the Streamer interface and ParseEvent are gone.
  • MaxTurns now requires a TurnLimiter, and Request.Schema a SchemaConstrainer. Both are refused before a process starts rather than silently dropped.
  • Result carries a json.RawMessage and is no longer comparable with ==; use reflect.DeepEqual.

Notes

Every fixture under testdata is real output from the two CLIs, sanitised. The decisions above are recorded in docs/adr/, and CONTEXT.md fixes the vocabulary — verdict, outage, refusal, unmet constraint — that means one thing regardless of which CLI is behind it.

The API is not yet stable.