Skip to content

v0.1.4

Latest

Choose a tag to compare

@aviggiano aviggiano released this 06 Oct 19:02
5b463c8

This release removes machinery and makes failures show up earlier. Campaign data governance and the sealed prompt-authority binding are gone. Ultrafuzz now deletes its own task worktrees after a run. The final report names the evaluated commit, adds remediation, drops low-level diagnostics and always shows a numeric spend estimate. Unsafe provider homes and reused run IDs are refused before planning, with a message that says what to fix. This is a breaking release: finish or cancel runs launched on an earlier release with that release, then rerun ultrafuzz init in each project after upgrading.

  • Less machinery. Data governance is removed. Where campaign data goes is now a documented trust assumption about the inference providers you configure. Every agent task gets its prompt input index from the live task plan, and the sealed authority selectors are gone, so a runtime-rendered prompt no longer names a file that was never written.
  • Cleaner disks and reports. With the default keep_workspaces = false, task worktrees are deleted once a run ends, and a rejected output is kept. report.md now shows the commit, a Remediation section for every issue and an Estimated spend that is always a number, with a trailing + when it is probably low.
  • Earlier, clearer failures. A provider home the agents would refuse, or a run ID the engine already knows, now stops run before planning and names the fix. resume --retry-failed no longer reports COMPLETE over output that no consumer read.

Breaking changes

  • [runtime] [artifacts] [modal] Removes campaign data governance. ULTRAFUZZ_DATA_GOVERNANCE_POLICY and ULTRAFUZZ_DATA_DISCLOSURE_ACKNOWLEDGEMENTS are no longer read. Campaigns have no private or public mode, and a target with uncommitted files, such as those ultrafuzz init scaffolds, no longer fails DATA_GOVERNANCE_PRIVATE_TARGET_UNBOUND. Route IDs, the per-adapter route re-check, data-governance.json and every DATA_GOVERNANCE_* diagnostic are gone. plan.json moves to ultrafuzz.run-plan.v4, and commands that read a run's sealed workflow refuse a run planned by an earlier release. ultrafuzz-modal runs private standalone benchmarks again. docs/security.md now says to review each provider's retention, training, region and DPA terms yourself. Thanks @aviggiano! (#1275)
  • [runtime] With keep_workspaces = false (the default), Ultrafuzz deletes task worktrees itself on the first sync after a run ends, instead of leaving it to the workflow engine, which kept almost every worktree. Each deleted worktree takes its Git registration and its ultrafuzz/<run>/<attempt> branch with it, along with test/foundry/ and foundry.lock. A worktree that holds a rejected output is kept. keep_workspaces is read from the configuration fixed at launch, so setting it for resume has no effect. Reads of a finished run's output, such as report, report bundle and the dashboard, check published files only. Thanks @aviggiano and @mrthankyou! (#1282)
  • [runtime] [artifacts] [prompts] Every agent task now gets its prompt input index, .ultrafuzz/authorities/<attempt-id>.json, built from the live task plan. Tasks that wait on a dynamic group, such as dedupe-findings and final-report, used to get no file, even though their prompt named one. The sealed prompt-authority selectors, their checks and the unread snapshot task manifest are removed, so this release does not read runs planned by an earlier release. Thanks @aviggiano and @mrthankyou! (#1285)
  • [runtime] [artifacts] [prompts] [cli] The final report is simpler and more actionable. report.md replaces Source run ID with - Commit: <hash>, taken from the run's source revision. It no longer renders the scoped coverage evidence and artifact validation warning sections; their data stays in report.json and the companion files. Every production issue ends with ### Remediation: the finding's own recommendation, or a fixed sentence saying none was recorded. Inline code in finding text renders as code. Estimated spend is always a number taken from accounting v4, with a trailing + when some usage was not recorded or could not be priced. Missing rates come from a small, dated fallback price table, and usage that was never recorded is never estimated. report@3 and run.json change in place, so reports written before this release no longer verify. EVMbench scores are not comparable across this release. Projects should delete .ultrafuzz/prompts/review/final-report.md and rerun ultrafuzz init. Thanks @aviggiano! (#1276)

Improvements

  • [runtime] [docs] validate, doctor and run warn with TOPOLOGY_TIMEOUT_SHADOWS_DEFAULT when a node or group timeout_seconds pin is shorter than the model profile or run.default_timeout_seconds it overrides. Before, raising a default silently did not reach a pinned node. The warning names the pin and, for a node pin inside a pinned group, the group pin it falls back to. Thanks @mrthankyou and @aviggiano! (#1256, #1273)
  • [runtime] The OpenRouterAgent Codex configuration allows up to 10 stream reconnects instead of 5. Codex reconnects a dropped stream inside the same session, so the agent keeps its earlier work instead of failing the node and starting over. The idle timeout stays at Codex's 5-minute default. Rerun ultrafuzz init to pick up the new adapter. Thanks @mrthankyou and @aviggiano! (#1277, #1280)
  • [docs] Recommends running campaigns from a published release that has been out for some time, rather than the unstable branch, and explains that a plain resume does not retry failed nodes. Thanks @aviggiano! (#1252)

Bug fixes

  • [runtime] validate, and so run, refuse an agent provider home that the agent adapters would refuse, before planning. The PROVIDER_HOME_UNSAFE diagnostic names the directory and the exact command that fixes it, such as chmod 700 ~/.codex. Before, a ~/.codex or ~/.claude with mode 0755 or 0775, as the CLIs create them, failed about two minutes into a run with an INTERNAL_ERROR that named no path. doctor reports the same diagnostic. The check also covers OpenRouterAgent, rejects a relative CODEX_HOME, CLAUDE_CONFIG_DIR or KIMI_* path, and falls back to the user's home directory when HOME is unset. Thanks @mrthankyou and @aviggiano! (#1267, #1281)
  • [runtime] ultrafuzz run --run-id <id> fails with RUN_ALREADY_EXISTS before planning when the workflow engine already records that run, as it does after ultrafuzz clean <id>. Before, it spent about 6 minutes and 1 GB planning, then failed at submission and left a partial run directory. The message suggests resume only when the run directory still exists. Thanks @mrthankyou and @aviggiano! (#1264, #1274)
  • [runtime] [modal] resume --retry-failed no longer reruns a failed failure_policy: continue task once a task that treats it as optional has already run without it. That rerun could make the report say COMPLETE over output no consumer read. A task whose consumer failed in preparation is still retried, and the Modal benchmark worker no longer starts a resume that has nothing to rerun. Thanks @mrthankyou and @aviggiano! (#1254, #1272)
  • [runtime] [ci] Moves production dependencies past new advisories: proxy-addr 2.0.8 (Critical), vue and @vue/server-renderer 3.5.42, source-map-js 1.2.2, prosemirror-view 1.42.3 and @modelcontextprotocol/sdk 1.31.0. The workflow engine that run installs now resolves its dependencies as of 2026-10-01, so it also picks up the earlier fast-uri and undici fixes. About 90 other engine dependencies move to newer in-range versions with it. The not-reachable http-cache-semantics advisory GHSA-ch52-4w7c-c8xp has an exception that expires on 2026-11-04 (#1278). Thanks @aviggiano! (#1279, #1283, #1286)

Full changelog: v0.1.3...v0.1.4