v0.2.1 — reproducibility record, and the fixes it forced

v0.2.1 — reproducibility, and the fixes it forced

v0.2.0 had one real run per request type. This release re-ran four request types three times each with the same request text and settings (local Claude Code CLI, claude-opus-5, $6 and 120 model calls per run), kept every result, and fixed what the series exposed. Evidence for all runs: docs/evidence/scenarios/rerun-2026-09-13/.

Three runs each, unfiltered

Request	Completed	Plan size	Review pass	Cost / run	Time / run
CLI tool in Python with unittest + README	3 / 3	2 tasks every time	22/22	$2.09–3.14	16–19 min
4-file static docs site	3 / 3	2 tasks every time	22/22	$1.62–1.72	15–17 min
Launch page + 3 post drafts	2 / 3	2 (one run grew to 6)	38/39	$1.70–5.67	20–37 min
Source-grounded comparison of 3 web pages	0 / 3 (all partial)	4–5	21/24	$5.48–6.11	31–35 min

The generated unit tests were re-run by hand for all three code runs (18 / 19 / 17 tests, OK); the three generated sites have no broken internal links.

What the series exposed, and what changed

• The default reviewer had no web_fetch. Asked to check claims against their sources, it could not; the Master added workaround tasks that exhausted the budget. All three research runs ended partial this way, each with the planned research.md already accepted. The reviewer now has web_fetch, and its prompt says to fetch cited sources (unverified if unreachable). One post-fix research run completed: reviewer fetched all three sources, 5/5 review pass, $2.05, 8m44s (one run, not a new series).
• Milestone replanning could start work it could not finish. Replanning is now skipped, and recorded as plan.milestone with skipped: limits, when less than one agent session of budget or model calls remains. The Master is told how many sessions remain and not to add polish once every deliverable is accepted (the one partial launch-page run was two rounds of wording polish after both planned tasks were accepted).
• Partial and failed runs carry a reason naming the unaccepted tasks, and say when every task of the original plan was accepted.

Also

• CI was red since v0.2.0 (the Docker sandbox test assumed ~/.cache); fixed. CI now runs the Docker isolation test on Linux and a headless-Chrome UI smoke of the bundled UI in English and Japanese on every push.
• agentteam probe prints a structured reason with a next step (claude CLI missing / not logged in / API key) instead of a traceback.
• Version comes from pyproject.toml everywhere (/api/health, UI header). CHANGELOG.md, feature-request template.
• 53 deterministic tests.

Install

uvx --from "git+https://github.com/FORIFOR/Multibot@v0.2.1#subdirectory=backend" agentteam quickstart

or pip install the attached wheel. Needs Python 3.12, uv, and either a logged-in Claude Code CLI or an API key for another provider.

Still open

Three runs per request is a record, not a benchmark. Research-heavy requests remain the most expensive. Local 7B models complete the mechanics but not the review protocol. No one outside the author has used it yet — issues welcome.

v0.2.0 — five real scenarios, Docker sandbox, one-command install, English UI

v0.2.0

Everything in this release was driven by real runs through the local Claude Code CLI (claude-opus-5) and two other providers; the unedited artifacts and event logs are under docs/evidence/.

Real scenarios (one run each, 2026-09-13)

Request	Result	Checks	Review	Cost	Time
Launch page + 3 post drafts	completed	10/10	6/6	$1.66	18 min
Python CLI + unittest + README (tests re-run in the Docker sandbox)	completed	14/14	7/7	$1.87	16 min
Source-grounded comparison of 3 pages	failed → partial after a scheduler fix	10/13	5/8	$0.91 + $6.07	5 + 24 min
4-file static docs site	completed	39/39	12/12	$3.03	15 min

Other providers on the launch-page request: OpenAI-compatible gpt-4.1-mini completed in 29 s ($0.02); local Ollama qwen2.5:7b reached partial; qwen2.5:3b could not plan.

New

• Docker sandbox (--network none, read-only root, host uid, cap-drop, resource limits); auto selection docker → seatbelt → refuse. Verified on a real engine (Colima).
• One-command install: uvx --from "git+https://github.com/FORIFOR/Multibot#subdirectory=backend" agentteam quickstart (UI, prompts, skills and schemas ship in the wheel; quickstart probes, serves and opens the browser).
• English UI with an EN/JA toggle.
• Milestone replanning: after the DAG finishes, the Master may add tasks if the goal is not met (max_replans).
• New checks: html_links, json_schema, python_syntax, file_size_max, regex_count.
• scripts/eval_scenarios.py to run request types end to end and record results; docs/config/cost-optimized.yaml.

Fixed (all found by real runs, all covered by tests — 46 deterministic tests)

• Reviewer verifying two tasks had only its last verdict applied
• Structured single-shot CLI calls hit max_turns
• Milestone-added reviewer tasks never started; reviewer ending without a verdict left the target stuck
• Diamond dependencies (t2→t1, reviewer→t1,t2) deadlocked
• Per-session budget cap failures now continue once; weak models that publish everything but skip finish_task are accepted with an explicit runtime note

MIT.

v0.1.0 — an AI team whose work you can audit

Agent Team v0.1.0 — an open-source AI team that ships real work, with a conversation you can follow

One request → Master plans → Researcher / Builder / Reviewer actually work → you get artifacts plus the real bot-to-bot messages, a timeline, and verification bound to each artifact revision.

Site + narrated intro (EN/JA): https://forifor.github.io/Multibot/

Real run (2026-09-13)

Through the local Claude Code CLI, no API key. Model reported by the provider: claude-opus-5. Request: Japanese launch page + 3 social-post drafts, stop before publishing.

• Deliverables: index.html, posts.md, HANDOFF.md, final-report.md
• 10 programmatic checks pass · reviewer 6/6 pass · 2 delivered messages · 39 model turns · $1.66 list-price · 18 min 37 s · completed
• Unedited artifacts and the 77-event JSONL log: docs/evidence/run4-* (earlier runs 1–3 included; run 1 exposed a reviewer bug, now fixed)

What's in this release

• Append-only event store (SQLite WAL, per-run seq, redaction) with chat / timeline / report as projections
• Real message delivery between bots (send_message → mailbox → message.sent); questions wake the other bot to answer
• Immutable, SHA-256 artifact revisions; checks and review verdicts bound to a revision; review → revise → re-review loop
• Runtime-enforced limits: tool scope, write scope, budget reservation, approvals (hash + nonce), cancel / resume / fork
• Per-bot connection, model, effort and lockable system prompt; configured vs provider-reported model both recorded
• Providers: claude_cli (local Claude Code, no key), Claude API (official SDK), OpenAI-compatible chat, Ollama; real capability probe before a run can start
• macOS seatbelt sandbox for builder commands; SSRF-guarded web fetch; sandboxed HTML preview
• React UI (request · run · settings), REST + SSE, CLI

39 deterministic tests pass. MIT. Design notes and references in docs/blueprint/.