Repository navigation
Benchmarks
The repository has three benchmarks. The numbers are in the files linked below, not here, so they stay in one place.
npm run benchmark runs benchmarks/benchmark.mjs on local fixture pages in headless Chromium. It counts the tokens in tool output (with tiktoken, o200k_base) and the time spent in the browser, and it measures tool schema size. Model time is not included. Read it as an upper bound on what an agent gains, not as an agent result.
Source and tables: README, Benchmarks.
npm run benchmark:e2e runs a fixed script against a Korean-language contest wizard and covers the commit-safety cases. It compares one action per call with a full snapshot after each, against inspect_compact plus one batch_actions call. It is a script, not an agent, so it must not be read as a comparison of agent stacks.
Results: benchmarks/E2E_COMPARISON.md.
benchmarks/e2e-agent/ gives a coding agent three tasks on local fixture sites (a job application with an upload, a catalog search over several pages, a booking with a draft step) and runs it with turbo-browser-mcp, agent-browser, browse and browser-use. The fixture server decides success.
Results: benchmarks/E2E_RESULTS.md. Plan and success checks: benchmarks/E2E_PLAN.md.
- All 12 stage-one trials (4 stacks, 3 tasks) succeeded, with one trial per cell and one host agent and model. That is not enough to rank the stacks.
- The ten-trial randomized protocol in the plan has not been run.
- Two stacks ran headless and two ran in a visible Chrome, so the times are not like for like. The token totals include cached context, which grows with the number of turns.
To run the agent benchmark yourself, see benchmarks/e2e-agent/README.md.