Skip to content

Releases

v0.3.0

Choose a tag to compare

@natsu0529 natsu0529 released this 20 Nov 16:53
26f27dc

This release makes the search itself better: an LLM judge joins the test suite as the value function, episodes can run in parallel, and a run can be resumed.

uv tool install agent-mcts    # or: uv tool upgrade agent-mcts

Highlights

LLM judge (--judge)

Tests alone stop being useful once they pass. On an already-green suite every attempt scores 1.0, the search ends after one episode, and an attempt that deleted the failing test looks perfect.

With --judge (or judge = true under [value]), each attempt is also scored by an LLM, as in SWE-Search's value agent. The judge reads the task, the attempt's cumulative diff and the test output, and returns a score plus a short critique.

  • The node's reward is w_test * tests + w_judge * judge (0.5 / 0.5 by default).
  • An attempt that changed nothing is not judged and gets a judge score of 0, so it can no longer tie with real work.
  • The critique is fed into the revision prompts of that node's children.
  • The judge runs through the same agent CLI as the search, in an empty scratch directory. judge_model lets you use a cheaper model for it.
  • Judge spend counts toward max_cost_usd but not toward max_nodes.

Two more uses of the judge:

  • Final comparison. When the search ends, the attempts close to the best score are compared head to head in one more judge call, and its pick is what apply offers. With a clear winner no call is made. Turn it off with select = false.
  • Sibling ranking (optional). With relative = true, attempts that share a parent are ranked against each other, and that standing counts in choosing which one to revise next.

Parallel expansion (-j)

-j 3 keeps up to three episodes in flight, each in its own worktree. On a small test task this cut a three-episode run from about 10.5 minutes to about 3.

  • Episodes in flight count toward max_nodes, and a launch that would cross the dollar ceiling is held back.
  • Your value command runs concurrently in different worktrees, so it must not depend on a fixed port or a shared file outside the worktree.

agent-mcts resume

agent-mcts resume [run-id] continues a run that was interrupted or ran out of budget. The tree keeps growing in place, under the settings the run was started with. -n and --max-cost set new totals for the run.

Selection policy knobs

New [search] settings, all off by default:

  • backup = "mean" | "max" | "mix"
  • selection = "global": pick the best expandable node anywhere in the tree, with optional depth_bonus and depth_penalty
  • give_up_after: stop revising a line after that many low scores in a row

Smaller changes

  • The pass ratio is now read from cargo test, go test and JUnit-style summaries, as well as pytest and jest.
  • A warning is shown when the baseline is already green and the judge is off.
  • agent-mcts show <node> prints the test and judge scores, the critique and the sibling standing.

Upgrading

  • Nothing changes unless you turn the new options on. The judge, parallelism and the policy knobs are all off by default.
  • Journals written by earlier versions still load. They do not record the run's settings, so resume on such a run warns and continues with the current .agent-mcts.toml.
  • v0.2.0 was tagged but never published to PyPI. If you are upgrading from 0.1.0 on PyPI, you also get what 0.2.0 added: the Codex CLI adapter, an episode-count budget for agents that report no dollar cost, and recovery of timed-out and interrupted episodes.

See the README for the full configuration reference.

Full changelog: v0.2.0...v0.3.0