ToolsEnabled Bench 0.3.2 (beta)

ToolsEnabled Bench 0.3.2 (beta) is a bug-fix and packaging release of the local benchmark builder and its MCP server. It fixes ways the web app could lose your work, gives clearer messages when local files are damaged or missing, and packages Bench as a Claude Code plugin. MIT licensed.

The runtime download keeps the name ToolsEnabled BenchMark Builder (toolsenabled-benchmark-builder-0.3.2.zip).

Data-loss fixes

• A project that fails to open is never overwritten. In 0.3.1, if a saved project couldn't be opened (for example, a draft an MCP client saved in a shape the editor couldn't load), the app could keep the previous project selected while showing the failed one's content, and your next edit saved that content over the previous project. Now a failed open saves nothing, leaves no project selected, and says why. When the failed project was opened from Runs with "Inspect results", the project picker no longer keeps showing the previous project.
• Routing edits stay put. In 0.3.1, changes typed into "Advanced: this routing as JSON" could be silently replaced by the next change in the routing tree or the rules table, even though the app said "Saved locally". The JSON box, the tree and the rules table now always share the same copy, so every edit survives autosave and reload. A related problem found during 0.3.2 testing, where a tree change could undo rules-table edits, was fixed before release.

Other fixes

• One damaged project file no longer hides the rest. If a project file can't be read or is corrupt, the app lists your other projects and shows how many files it skipped. Error messages no longer show full file paths from your computer. A failed save keeps your last saved version and leaves no temporary files behind.
• The second-Inspect issue from 0.3.1 is fixed. If a second Inspect fails, the first inspected study no longer stays selected, so Submit can't run it by mistake.
• Your data folder must be outside the Bench folder. The web app and the MCP server now refuse to start without an absolute BENCHMARK_DATA_DIR outside the Bench folder, instead of quietly creating data inside it. The message names both BENCHMARK_DATA_DIR and the Claude plugin's data_directory option.

New: Claude Code plugin packaging

• The ZIP is also a Claude Code plugin, toolsenabled-bench, with the 12 Bench tools and a workflow skill. On install it asks for a private data folder (data_directory), which must be outside the plugin's install and cache. See docs/CLAUDE-INSTALL.md.
• No npm install when you install the plugin. The runtime ships no dependencies, package lock or install scripts, so installing or starting it runs no npm and needs no network.
• The same folder also carries a Codex plugin manifest. Codex loads it with all 12 tools once BENCHMARK_DATA_DIR is set.

Unchanged: the 12 MCP tools and their schemas, execution confirmation (confirm = the exact study ID, plus trust: true for studies frozen elsewhere), the one-process-per-data-folder lock, and the print-only setup helper (node tools/mcp-config.mjs --client claude|codex|cursor|claude-desktop|deepseek), which now needs BENCHMARK_DATA_DIR set first.

Run locally

1. Download toolsenabled-benchmark-builder-0.3.2.zip and toolsenabled-benchmark-builder-0.3.2.zip.sha256, then run sha256sum -c toolsenabled-benchmark-builder-0.3.2.zip.sha256.
2. With Node.js 22.19 or later, extract the ZIP, then from its folder run node tools/release.mjs --verify.
3. Pick a private data folder outside the Bench folder, for example export BENCHMARK_DATA_DIR=/absolute/path/bench-data.
4. For the web app: node server/main.mjs, then open http://127.0.0.1:4318.
5. For your agent: node tools/mcp-config.mjs --client <your client> and add what it prints.

Only one process can use a data folder at a time. Stop the web app before an agent uses the same folder, and give agents that run at the same time their own folders.

Upgrading from 0.3.1: 0.3.1 kept data in .benchmark-data inside the Bench folder by default. Move that folder somewhere outside the new Bench folder, set BENCHMARK_DATA_DIR to it, and run the setup helper again. Your projects and frozen studies come along: they list, export and analyze in 0.3.2. To qualify or run a study frozen with 0.3.1, use 0.3.1, because running a study needs the exact runtime it was frozen with, or freeze a new copy in 0.3.2.

Tested on this exact release (source tag v0.3.2, commit 04cf89d; runtime ZIP SHA-256 94923d535c5929013928d20c11e1a81a81a219bb96f74b03be60f37f73cd9502)

• 548 of 548 tests passed headless, plus 53 real-browser regression checks and the full browser journey with no page errors.
• Clean builds from the reviewed source and from this public source produced byte-identical ZIPs.
• Real agents, through MCP only: Codex ran the offline recorded example end to end with Bench's tools: edit the composition, freeze, export, qualify, run, analyze and report. It produced study export SHA-256 92c4dbd049b8a5f474c3ed85cac6d79e0a3c12f3c4ba86af7926e3a25a111327, with 2 of 2 recorded tasks completed and passed. Claude Code 2.1.287, with the ZIP installed as a plugin and an absolute external data folder, ran the same example: 9 Bench calls, 2 of 2 tasks passed, and a byte-identical export. A DeepSeek Harness run on this exact ZIP got through freeze and export with the same export hash, but was cut short by an unrelated sandbox restart before qualify and run, so it is not counted.
• Codex plugin loading, no sign-in: in a fresh, empty Codex profile, Codex installed the ZIP as a plugin and listed all 12 tools. With the data folder missing or not passed through, it started no tools and created no data.
• Independent security reviews: a full review of the 0.3.2 changes, two follow-up reviews and a final check. Fixes along the way include both data-loss bugs above, file paths in error messages, and data written inside the install folder. The final check found nothing at Medium severity or above.

Limits

• Local only. ChatGPT needs a hosted server with sign-in, which is a later phase.
• The Claude Code marketplace entry isn't published yet. Until it is, register Bench with node tools/mcp-config.mjs --client claude.
• The Claude Desktop bundle (.mcpb) isn't part of this release. Native Desktop install on macOS and Windows still needs testing and production signing.
• This release was tested on Linux. macOS and Windows weren't re-tested.
• The installed Codex plugin passes only BENCHMARK_DATA_DIR to Bench, so it suits recorded and offline studies. For a study that calls a provider, use a direct registration and forward the key's variable name yourself, as docs/CLAUDE-INSTALL.md shows.
• Codex asks for approval before Bench's write and execution tools. For non-interactive codex exec, grant it with Codex's own per-server approval setting.
• The included examples are authored, recorded controls. This release contains no live model runs, model scores or benchmark results.
• A checksum shows the content is identical; it isn't a publisher signature or an independent scientific validation.
• Running a study from someone else executes its code with your account's permissions. Run only studies you trust.
• Known issue (Low, fix planned): the web app skips a damaged project file, but the MCP projects.list tool still fails until that file is fixed or moved out of the projects folder. Opening healthy projects by ID still works.

0.3.1, 0.3.0, 0.2.0 and 0.1.0 exports keep working with their own pinned runtimes.

ToolsEnabled is not affiliated with or endorsed by Anthropic, OpenAI, DeepSeek, Cursor or NVIDIA.