Skip to content

v0.7.0 — Ollama-backed ability tests

Choose a tag to compare

View all tags
@pluginslab pluginslab released this 13 Mar 21:39
· 355 commits to main since this release
v0.7.0
a465729

What's changed

The ability test runner has been completely rewritten to use a local Ollama instance instead of Puppeteer + WebGPU. This makes tool selection tests dramatically faster and more portable.

Highlights

  • 20s test runs (was 5+ minutes with the browser-based harness)
  • No browser needed — pure Node.js, no Puppeteer, no WebGPU, no Chrome
  • No webpack build step — npm run test:abilities runs directly
  • Auto-setup — runner installs Ollama via Homebrew and pulls qwen3:1.7b on first run
  • Same system prompt — mirrors react-agent.js exactly, so results are directly comparable to browser
  • Baseline established — Qwen 3 1.7B: 100% tool selection accuracy (8/8 core tests)

Breaking changes

  • Removed puppeteer devDependency
  • Removed src/extensions/test-harness.js and tests/abilities/test-page.html
  • Removed build:test-harness npm script
  • Simplified webpack.config.js (no more test-harness entry point)
  • npm run test:abilities now requires Ollama instead of Chrome with WebGPU

Running

npm run test:abilities -- --file tests/abilities/core-abilities.test.js

Options: --model qwen3:1.7b, --no-think, --verbose

Full changelog

a465729...69b8621

Assets 2
Loading