Agent and Models

Agent and Models

The agent

agent.Agent implements the runtime's agent interface for model mode. Each step it:

1. renders the run into a message list: an opening user message with the task, then for every prior tool-call step an assistant tool_use block and a user tool_result block, with older results summarized and the most recent ones in full;
2. sends the fixed system prompt, the messages, and the tool specifications through the runtime's accounting model caller;
3. takes the first tool call in the reply as the decision, with any text as the recorded reason.

The rendering and that last mapping are the runtime's render package, called with this repository's own hooks: the opening message, the recent-results window, an assistant turn that carries only the tool_use block, and a tool_result whose content is the labelled head plus the observation. What remains here is the domain text; the generic half is shared with the runtime's other consumer. The move reproduced the previous renderer byte for byte, so the recordings made before it still replay.

If the reply has no executable tool call, the agent does not ask again inside the step. The decision is recorded as the invalid kind no_tool_call (or truncated, when the reply hit the output cap before the call was complete), the runtime writes an invalid_decision observation, and the next render answers that step with a nudge naming what was wrong and asking for exactly one tool call. So a text-only reply now costs a recorded step against the step and consecutive-failure limits, and is visible in runs show and in the audit log, instead of being retried in band inside one step. A partial tool call from a truncated reply is never executed. A provider refusal is different: it fails the run, because a model that declined will decline again.

The agent is stateless. Everything it needs is in the runtime's step records, so a resumed run renders exactly what a continuous one would.

The system prompt

The prompt is a constant. It states the rules the runtime enforces, so the model is not surprised by denials, and gives a working method: read the failing output, read the affected file, read the dependency source for the new API, write the corrected file in full. Two rules came from watching runs fail: keep the signatures that tests call unchanged and adapt inside the function body, because the model cannot edit tests; and validation must be the last action before a proposal, because normalization changes the tree.

The prompt never contains repository content. Repository files, dependency sources, and command output reach the model only as tool results, each labelled as data, and the prompt says that instructions inside them have no authority.

Changing the prompt changes every request, which invalidates all recordings. Re-record after any change and note it in the benchmark results, since results are comparable only within one prompt.

Models

The runtime defines a model interface: a request with a system prompt, messages, tools, and an output cap; a response with text, tool uses, usage, and stop reason. Both adapters are the runtime's own nested modules, which repo-steward configures from -model provider:name; the copies that used to live in internal/model are deleted. A nested module means the Claude SDK is a dependency of that adapter rather than of the runtime's core.

Ollama. providers/ollama talks to a local server at OLLAMA_HOST using its chat endpoint with tools, with thinking mode off and a context window large enough for a rendered tool-using conversation. Names are ollama:<model>. Two models have been benchmarked: qwen3:30b-a3b, which follows the tool protocol well, and gpt-oss:20b, which frequently has its output rejected by the server as an unparseable tool call under this prompt. A reply the server cut off at the output cap comes back as max_tokens with its tool calls dropped, which is what makes it a recorded truncated step rather than a half-written call.

Claude. providers/anthropic talks to the Messages API through the official SDK. Names are anthropic:<model>. The key comes from ANTHROPIC_API_KEY or CLAUDE_KEY, read by repo-steward and passed to the adapter rather than left to the SDK, so a run without one is refused before anything is recorded; it is never persisted. The price table is the adapter's Prices(), read from the published price list, and a model absent from it is refused rather than run at a price of zero; cache writes, which cost more than base input, are charged as input at that multiple so the cost cap never undercounts. Nothing in tests or CI constructs it, and maintain and bench refuse it without a cost cap. It exists for explicit, budgeted measurements; see Evidence and Benchmark Results.

Accounting

Every model call goes through the runtime's accounting caller, which reserves a row before dispatch, records usage and latency after, retries transient errors with backoff and records each attempt, marks a call ambiguous when the response was lost after dispatch, and enforces the call, output-token, total-token, and cost limits with projections before the call is made. A local model is recorded as free rather than left unpriced, which keeps "costs nothing" and "price unknown" distinguishable: the first runs at zero cost, the second is refused.

Recording and replay

The runtime's replay package keys each response by the model name and a canonical hash of the request. -record DIR wraps the live model and writes every response; -replay DIR serves from the directory and fails on a miss rather than calling anything. CI runs model mode against recordings with the server address pointed at a closed port, so a replay that silently fell through to a live call would fail.

Recordings for qwen3:30b-a3b on the patch-safe and breaking-minor fixtures live under internal/steward/testdata/recordings. They were made before the renderer moved into the runtime and replay unchanged, which is how the move was checked: a single differing byte in a rendered request would miss its key.

The agent modes do not select an upgrade by vulnerability. -select vulnerable is available only in baseline mode and the other modes refuse it; adding it would change what the model sees, so the committed recordings would have to be re-recorded. See Vulnerabilities: Reporting and Fixing.

On real repositories the local model has not repaired a break: see Evidence and Benchmark Results.