Skip to content

v0.7.4 — 'act, don't narrate' guard

Choose a tag to compare

@hackspaces hackspaces released this 10 Jul 18:27

A harness fix surfaced by running forge through a real, standard agentic coding benchmark (bare vs full harness ablation).

What the benchmark caught

Running the same model (qwen3-coder:30b) with the harness off vs on, the full harness scored worse (0/5 vs bare 2/5) — not because the model was worse, but because of a harness defect: the model would read the files, emit a say like "I'll implement it… let me analyze", and the harness accepted that as the finished turn and stopped — before writing a single line. The done-gate only bounces a say after file mutations, so a read-then-narrate turn slipped through.

The fix

In autonomous mode, a say that only describes upcoming file work after a zero-mutation turn is now bounced once — "do the work, don't narrate your plan." This took the full harness from 0/5 → 3/5 on the slice, eliminating the regression. It also helps small models, which are especially prone to narrate-and-bail.

This is the "test the harness, not the model" methodology in action: an ablation on a real benchmark found a genuine scaffolding bug that a raw pass-rate would have hidden.

Stdlib-only, zero runtime deps. 423 tests green.