Vision and Philosophy

Vision and Philosophy

The thesis

kstrl exists to answer one question: how far can a single operator push autonomous software production without giving up trustworthiness?

Most agent harnesses optimize for generation. kstrl optimizes for verification, and it is built as a loop that closes on evidence rather than on the agent's opinion of itself. The agent writes code and says it is done. kstrl treats that as a claim, measures the work with checks the agent did not write, turns the gap into the next instruction, and stops only when the independent checks agree. Every part of the system exists to serve one of those steps, and the parts were chosen together rather than accreted.

The roles inside the loop are deliberately adversarial. The architect red-teams the spec before any code exists, the reviewer hunts for requirement violations and scope creep, the security reviewer works from an OWASP-mapped threat taxonomy. Adversarial framing causes a model to find defects that a helpful-assistant framing glosses over. When editing any role, the standing question is: "does this make the role more skeptical, or more eager to please?" Prefer the former.

Three commitments follow:

1. Calibration over claims. A reviewer's self-report ("exhaustively searched") is a hint, never a signal. The calibration suite plants known bugs and measures each adversarial role's detection rate against a saved baseline. A prompt edit without a calibration check is treated as untested.
2. Halt over heroics. The architect halts on blocker-severity spec issues; hard-mode reviewers halt at their thresholds; a run that reaches a bound stops loudly rather than degrading silently.
3. Audit trail. Every decision writes a record: findings to PR bodies, knowledge facts to disk, outcomes to the evolution journal. If it is worth deciding, it is worth recording. The filesystem is the event bus; every UI is a projection of events.jsonl.

The loop, in plain terms

Think of a thermostat. There is a temperature you want, a thermometer that reads the room, the difference between the two, a box that decides what to do about the difference, and a boiler that acts. What makes it work is that the thermometer is not the boiler. If the boiler reported the temperature it would always report success, and the room would freeze.

kstrl has the same shape and the same rule.

• What "done" means is written down before work starts: acceptance criteria on each user story, a policy envelope on each merge, a floor on test quality. Those are the target the whole loop steers toward.
• What was actually built is measured by things the agent did not write: the test suite, the type checker, the linter, a scope check on the diff, a reviewer from a different model family, a security reviewer, integration tests across components, and calibration, which is the check pointed at the other checks.
• The difference between the target and the measurement is a typed finding with a file, a line, an explanation and a suggestion. That finding, not the raw tool output, is what the agent receives next.
• The decision about what to do with the difference (retry, halt, merge, or lower the factory's own autonomy) is made by the harness, in code, outside the agent's reach.
• Disturbances are everything the loop did not command: a model that answers differently to the same prompt, a push that fails, a base branch that moved, a teammate's commit. The design names them so they can be told apart from real signal: an infrastructure failure is recorded as a finding of its own kind and never counted as a lesson.

There is more than one loop, and they run at different speeds. The engineer iterates on a story in seconds. A component is accepted or retried in minutes. Components are merged and integration-tested in tens of minutes. The daemon admits new work over hours. The factory's own permissions move over days, and what it has learned moves over weeks. Each loop's output is the next slower loop's target, and the slower loop only acts on what the faster loop has already measured.

What the factory refuses to trust

The design is easiest to understand through what it will not accept as evidence.

• The agent's word. The story flag it sets, the summary it writes, the completion marker it emits: all claims. The mechanical checks, the reviewer's per-criterion verdicts and the test suite are the measurement.
• A reviewer's word about itself. "Searched exhaustively" is rendered in the PR body and never gates anything. Only planted-bug calibration says whether a reviewer works.
• A green test suite on its own. Agent-written tests can assert nothing falsifiable, and an agent that is rewarded for passing tests has every reason to weaken them. The adequacy gate reads the diff for deleted tests, added skips and assertion-free test files, and the approved fixtures are input-output pairs held outside the tree the agent can write.
• A phase that did not run. A skipped or crashed reviewer leaves a finding that says so. An empty findings list means every check ran and found nothing, never that nothing looked.
• An invented number. No threshold is chosen to make a feature shippable. A gate that cannot yet be measured ships in advisory mode, recording what it would have blocked, and graduates to blocking once the operator has read its output on real runs and can say what it caught and what it flagged wrongly.

Why 100% dark is a non-goal

A software factory with no human in it at all is not the target, and the reasons are structural rather than cautious.

• Oracle collapse: a fully closed loop in which one model certifies another has no independent oracle, and a model's preference for its own output strengthens as it gets better at recognising it.
• Shared blind spots: when the builder and the verifier share a model family, their errors correlate. When two such models are both wrong, they agree far more often than chance. Cross-family review at high autonomy exists because of this.
• Test gaming is measured behaviour, not a hypothetical. Green tests alone are not a merge gate at high autonomy; hence oracle linting, patch coverage, diff-scoped mutation testing and spec-derived fixtures.
• An operator who never intervenes loses the ability to intervene. kstrl keeps sampled human review at every autonomy level so the operator's judgement stays calibrated.

Where the human stands

There are three places a person can be. In the loop: a required step, and nothing proceeds without them. On the loop: the system runs itself, the person watches, intervenes on exception, and adjusts its behaviour between runs. Out of the loop: absent.

kstrl puts the operator on the loop at every level and gives them a named channel at each one. Authorising work by labelling an issue. Writing what a good change looks like in this repository, and keeping standing corrections in one file the agent always reads last. Running any check by hand on any tree. Approving or rejecting a merge. Promoting the factory's autonomy with evidence and an acknowledgement, and reading every demotion in one inbox. Leaving a comment on a pull request that reaches the next run. Boundary conditions come to the operator; everything else flows.

Autonomy is earned, bounded, revocable

The right to ship without per-change sign-off is not configured; it is earned by demonstrated compliance, bounded by a written envelope, monitored continuously, and revoked automatically with a defined path back to human gating. That is the model behind the autonomy ladder: four ordered levels, promotion on evidence plus a recorded acknowledgement, demotion one level per trigger with a cooldown, and a permission bundle derived fresh at every run start that can only withhold, never grant.

Two older lessons from factories of any kind earned their place alongside it. A factory without demand flowing through it is a demo, which is why continuous intake is treated as a survival capability rather than plumbing. And quality assurance is the last thing to automate away: even the most automated plants keep people for it, and every autonomy level here keeps a human-takeover path.

Doctrine

The standing rules that shape every design decision:

1. Integrate at the edges, build only thin middles. Queue front-ends, deploy engines, error tracking, notifications: integrate existing machinery. The state machines, policy checks and gates that carry trust: build them small, harness-side, and mechanically verifiable.
2. Enforcement reads artifacts, never agent self-report. Policy and adequacy checks run on the git diff, lockfiles, and tool output, in the mechanical verifier, outside the agent's reach.
3. Autonomy is earned, bounded, revocable. Promotion requires evidence plus a recorded human acknowledgement; demotion is automatic; fast down, slow up.
4. No assumed numbers. Every threshold is a placeholder until it has been produced by a gate running in advisory mode or replayed against recorded run data. Measure, do not estimate.
5. The human is a role, not a bottleneck. Spec-blocker halts, the optional pre-merge checkpoint, the exception inbox: boundary conditions route to the operator; everything else flows.
6. Every component runs by hand first. A check that can only be reached by starting a full factory run cannot be tuned, cannot be trusted cheaply, and cannot be debugged. Each part of the loop is a command the operator can run on its own before it is wired into automation.

Where this is going

The end state the current cycles aim at: specs, features and production bugs flow into a queue; the factory builds, verifies, merges, releases and observes what it shipped; runtime regressions re-enter the queue with reproducing evidence; the factory's own health governs how much autonomy it is allowed; what it learns in one project reaches the next, and a lesson that stops helping retires itself; and the operator handles an inbox of genuine exceptions and a handful of standing corrections instead of babysitting runs. The design that names each of those loops and what closes them is docs/control-loop-design.md; the concrete plan is on the Roadmap.