Repository navigation
v0.1.0
quality-loop 0.1.0
The first release. quality-loop measures what an agent-facing product teaches:
a skill bundle, an MCP server or a CLI, used by an agent that has nothing else.
Each task runs in a room that holds the product's agent-facing surface and
nothing more, started so it cannot read the repository the answers are in. A
second agent, the operator, puts up the screen the task needs. Afterwards it
reports the problems it found, each with a severity. There is no pass/fail.
The report is for a human to act on.
- Scenarios are Markdown. TOML front matter, then
## Prompt,## Screen,
## Mustand## Must not. The prompt is copied into the room verbatim by
code, never by an agent. A scenario whose prompt carries one of its own Must
lines is refused, because that is the scoring key leaking. - Rooms are isolated by flags and checked by an audit.
--restricted, an
explicit permission list underdontAsk, exactly the product's MCP servers,
git stopped at the room, and stdin closed. Afterwards the transcript is
audited for reaches into the checkout, the installed package, the user's
set-aside files, or another author's work. A refused reach is reported. Only
one that came back with something is a breach. - The user's files are stood in for and always put back. Fixtures replace
the product's files for each room. Everything the room made is collected as
evidence, and the originals return in afinallythat Ctrl-C and SIGTERM
unwind through. A stand-in the room could reach refuses the room. - The operator has no shell. Its tools are Read, Glob, Grep, writes to its
own directory and the room'sREPORT.md, and quality-loop's own MCP server.
That server opens and closes screens the product declares by name, and
runs the product's declared evidence commands with no shell. It never uses
the product's MCP, so a broken product cannot hide a failed setup. - Screens on macOS and Windows. Pages are served over a local HTTP origin
and record what is done on them. Terminal scripts are Python, run in a fixed
shell without the user's startup files. On macOS this uses Safari and
Terminal. On Windows it uses Edge and Windows Terminal; the Windows backends
have not yet been run on a Windows machine. - Reports feed a loop. Each problem has a stable id, a kind (skill gap,
product, harness, out of scope), its evidence, where it would be fixed, and
how a fix is confirmed.FINDINGS.mdandfindings.jsongather every room,
most severe first.
examples/toy/ runs the whole thing against a product small enough to read
in a minute. Keyhac is the first real product (crftwr/keyhac#205).