Repository navigation
v0.5
Forge now picks models from measured results rather than a decision model's opinion: it explores your own pools by default, compares models and efforts per tier with forge dataset routing, moves a task to a stronger model after three failed attempts, and flags tasks too large to pass review first time. Jev's effort sizing, tier rewriting and calibration are gone, and Cloudflare's hosted Clef can answer chosen Jev decisions instead.
Added
- With a Cloudflare provider configured, forge records two predictions it never acts on: whether each task's first attempt will pass review (once per plan, after models are assigned) and whether a retry will pass with the findings it was handed. They are written beside the task (
clef.first_pass,clef.retry_pass.<n>) and into the run dataset, and the export attaches them to its routing and retry cases, so their accuracy on new runs can be measured before anything relies on them. No prompt, verdict, table or retry ever sees them;FORGE_JEV_OUTCOMES=offturns them off. - Forge can send chosen decision sites to Cloudflare's hosted Clef instead of TypeSafe's Jev:
forge jev setup --provider cloudflare --account-id <id>validates a Workers AI token (kept incloudflare.envbesidejev.json, never in argv or logs) and maps the sites a paired replay of forge's own live requests showed Clef no worse on, which today is drift prediction alone;forge jev sitesshows or changes the map andFORGE_CLEF=offsends everything back. Thresholds can now differ per model, andstatusanddoctorreport readiness per site and provider. Without a Cloudflare provider configured nothing changes. - After three failed attempts on one model, a decomposed task's next retry goes a tier up in the pools you gave
plan, instead of repeating a dispatch whose fourth try passes least often. When no stronger model exists, the task stops with a split suggestion, and a plainretryrefuses until you split the task or name a model with--dwarf. Use--escalate-after Nto change the threshold, or--escalate-after 0to turn this off. plannow names tasks that are large enough that their first attempt usually fails review (at least 7,000 characters of prompt or 9 declared files; forge's history passes about a third of those), and tasks whose prompt has no## Acceptance criteriasection. The note is advisory: the table and the exit code are unchanged.
Changed
plannow explores by default (--explore 0.2): now and then, a row left to forge gets the model one tier away in your own pools (cheaper for medium and high tasks, stronger for low ones), with the odds recorded. The newforge dataset routingcommand compares each pair of models from those draws, and also shows every model's first-pass rate and runtime per tier. Pools that differ only in effort (--dwarf-low sol --dwarf-high sol:xhigh) are compared too, so the table also shows whether a higher effort passes more.--explore 0turns exploration off, and that choice is kept for later re-plans.
Removed
- Breaking: Jev no longer suggests efforts or rewrites tiers, because forge's run history showed it could not choose either: its effort suggestions carried no information beyond the task's tier.
forge jev calibrate,forge jev parse-specandforge jev qa-effortare gone, and so are the effort suggestions in Jev's routing lines and Fractal's re-rating of child tasks.--jev-actis still accepted but changes nothing.forge jev doctorlists therouting_actandcalibrate_floorthresholds as safe to delete from an existing config. Drift checks, memory curation, verification and the other advisory gates are unchanged.
Fixed
- The run dataset no longer drops an event when several writers in one process are the first ever to write to a new store: switching the new file to WAL is no longer raced by every connection, and losing that race is harmless.
- A comma list in
--qa(or--qa-high,--qa-medium,--qa-low) now rotates through its reviewers task by task, as the docs promise and as the dwarf flags always did. Before,plancopied the whole list into every row, so every review failed before reaching a model. A list typed directly into a task's dwarf or qa column is now refused when you runplan.
Security
- A decision-model API key or token that straddled the 200-character cut of an error message could leave its first characters in
jev.jsonlandjev.skip; error text is now redacted before it is shortened.