Repository navigation
Two findings now hold a close once by default, Cursor gets what it was missing, and the bench says what it measured and how. Handoff holds a blocked close after a search that found nothing, and hygiene a block that contradicts itself - the collection's two exceptions to never blocking by default. On Cursor, epistemic and persistence see failed commands again, executive names a plan that changed on disk, and the bench's node fence holds; Factory Droid gets a marketplace index. Executive answers its revision question by what a part does, not by its name. Benchmark 1.2.0 stops handing the arm without the plugin the behaviour it scores, and the report shows each plugin's gain over that arm, per case and per model.
| Plugin | Version |
|---|---|
| executive-self-monitoring | 1.11.0 |
| epistemic-self-monitoring | 0.6.0 |
| persistence-self-monitoring | 0.5.1 |
| termination-self-monitoring | 0.5.0 |
| coverage-self-monitoring | 0.7.1 |
| handoff-self-monitoring | 0.8.0 |
| progress-self-monitoring | 0.10.0 |
| integrity-self-monitoring | 0.4.0 |
| aspiration-self-monitoring | 0.4.0 |
| hygiene-self-monitoring | 0.4.0 |
Added
- Factory Droid:
.factory-plugin/marketplace.jsonlists the ten plugins, and the hooks count Droid's tool names (Create,ApplyPatch,Execute). Not run on Droid yet: no licence is available here. - executive 1.11.0: on Cursor, a plan that changed on disk is named with the next tool result. Cursor has no prompt hook that can add context; a
postToolUsehook keeps the same record as on Claude Code and says the same line after the first tool call that follows the change. With the plugin, Composer 2.5 went from 0/3 to 3/3 onexecutive/the-plan-that-changed. - The bench report shows each plugin's gain over the arm without it, in points, per case, per model and on each control; a case whose host limits what it can measure carries a note.
- The Cursor outcome driver re-scores stored runs (
--rescore) and checks the plugin code each run loaded. - A case no stage can score can be reported as not scored (
bench.js --not-scored), with its reason: the model is pooled over the cases it has, and the page says how many. - CI runs on macOS beside Ubuntu and Windows.
Changed
- handoff 0.8.0 and hygiene 0.4.0: two findings hold the close once by default - the collection's two exceptions to never blocking. handoff: a
blockedclose, or aneeds-decisionthat asks whether to do the work, after a search that found nothing. hygiene: a block that contradicts itself - Outside names a change the Decision keeps, or the Decision puts a change back while the code still has it - with a third way out: what the request needs, shown by a run, moves to Request names. - handoff 0.8.0: a fact the agent lacks is not a block.
blockedis a limit that stops the output itself; a missing fact goes in as a named assumption with the default. - hygiene 0.4.0: "needed" is a run, not an argument - a function the request does not name is needed only if a fix that changes only what it names fails the request's own example.
- executive 1.11.0: the revision question is answered by effect. A part reached under another name still does its work if the result changes without it, and
revise-planis a question for the owner, not a stop to the work they asked for. - epistemic 0.6.0 and persistence 0.5.1: on Cursor a failed command reaches them (
postToolUseFailure). epistemic also watches every tool that returns a command's output, and asks about a marked conjecture after a result that bears on it. - Benchmark 1.2.0: three canonical cases stop handing the arm without the plugin the behaviour they score, and a bench stage runs the canonical cases, one per plugin, unless told otherwise.
Fixed
- The bench's node fence holds on Cursor and in Git Bash. Cursor puts its own
node.exefirst on its shell's PATH; a fenced run now starts the CLI from its version folder with.EXEout of PATHEXT. Git Bash did not find the fence'snode.cmdby its bare name. No stored score rested on a run that used either gap. - The audit reads more: a run's own session file, a write outside its workspace, Cursor's refusals and chats, and Git Bash's
/tmp. - A stage keeps what it scored: a finished stage with an unscored case keeps the rest, and a stage stopped at the quota window keeps its re-run marks.
- Graders: the handoff cases read seven more ways of naming a default; the epistemic case reads a block's fields on one line and a cause stated beside the one it rules out; the header case and the weekly report were fixed in a review of the first canonical session.
- The report's figures come from the runs, with a Failed runs section that lists what each grader found.
- coverage 0.7.1 and persistence 0.5.1: the code no longer describes the evaluation.
- Two failures the first macOS run found: a cut JSON report, and a fence that denied the workspace reached through a link.
- A file that arrives between two sessions lands dated now.
Full Changelog: v0.19.0...v0.20.0