v0.2.1 — reproducibility record, and the fixes it forced

v0.2.1 — reproducibility, and the fixes it forced

v0.2.0 had one real run per request type. This release re-ran four request types three times each with the same request text and settings (local Claude Code CLI, claude-opus-5, $6 and 120 model calls per run), kept every result, and fixed what the series exposed. Evidence for all runs: docs/evidence/scenarios/rerun-2026-09-13/.

Three runs each, unfiltered

Request	Completed	Plan size	Review pass	Cost / run	Time / run
CLI tool in Python with unittest + README	3 / 3	2 tasks every time	22/22	$2.09–3.14	16–19 min
4-file static docs site	3 / 3	2 tasks every time	22/22	$1.62–1.72	15–17 min
Launch page + 3 post drafts	2 / 3	2 (one run grew to 6)	38/39	$1.70–5.67	20–37 min
Source-grounded comparison of 3 web pages	0 / 3 (all partial)	4–5	21/24	$5.48–6.11	31–35 min

The generated unit tests were re-run by hand for all three code runs (18 / 19 / 17 tests, OK); the three generated sites have no broken internal links.

What the series exposed, and what changed

• The default reviewer had no web_fetch. Asked to check claims against their sources, it could not; the Master added workaround tasks that exhausted the budget. All three research runs ended partial this way, each with the planned research.md already accepted. The reviewer now has web_fetch, and its prompt says to fetch cited sources (unverified if unreachable). One post-fix research run completed: reviewer fetched all three sources, 5/5 review pass, $2.05, 8m44s (one run, not a new series).
• Milestone replanning could start work it could not finish. Replanning is now skipped, and recorded as plan.milestone with skipped: limits, when less than one agent session of budget or model calls remains. The Master is told how many sessions remain and not to add polish once every deliverable is accepted (the one partial launch-page run was two rounds of wording polish after both planned tasks were accepted).
• Partial and failed runs carry a reason naming the unaccepted tasks, and say when every task of the original plan was accepted.

Also

• CI was red since v0.2.0 (the Docker sandbox test assumed ~/.cache); fixed. CI now runs the Docker isolation test on Linux and a headless-Chrome UI smoke of the bundled UI in English and Japanese on every push.
• agentteam probe prints a structured reason with a next step (claude CLI missing / not logged in / API key) instead of a traceback.
• Version comes from pyproject.toml everywhere (/api/health, UI header). CHANGELOG.md, feature-request template.
• 53 deterministic tests.

Install

uvx --from "git+https://github.com/FORIFOR/Multibot@v0.2.1#subdirectory=backend" agentteam quickstart

or pip install the attached wheel. Needs Python 3.12, uv, and either a logged-in Claude Code CLI or an API key for another provider.

Still open

Three runs per request is a record, not a benchmark. Research-heavy requests remain the most expensive. Local 7B models complete the mechanics but not the review protocol. No one outside the author has used it yet — issues welcome.