Skip to content

v0.3.0 — 7B Models, E2E Test Suite, UI Improvements

Choose a tag to compare

@pluginslab pluginslab released this 02 Mar 14:15
· 380 commits to main since this release

What's New

7B Model Upgrade

  • Default model: Qwen2.5-7B-Instruct — 96% E2E accuracy, 100% JSON reliability
  • Available models: Qwen 2.5 7B, Hermes 2 Pro 7B, Llama 3.1 8B
  • ~5GB VRAM, ~4.5GB download (one-time, cached via Service Worker)
  • Up from 74% accuracy with 3B models in v0.2.0

E2E Browser Test Suite

  • 27 tests across 8 categories (L2 basic + L3 advanced agentic)
  • Validates full pipeline: user message → LLM reasoning → tool selection → tool execution → response
  • Chrome DevTools MCP integration for Claude Code
  • Test observability hook (window.__wpAgenticTestHook)
  • 4 workflows tested at 100% pass rate

Bug Fixes

  • JSON envelope ({"action": "final_answer", ...}) no longer leaks into user-visible responses
  • Loading spinner ("Thinking..." / "Running tool...") appears inline in message flow after user's chat bubble
  • Spinner dot properly aligned with timeline dots

Documentation

  • Updated all docs (README, ARCHITECTURE, IDEAS, CONTRIBUTING) for 7B models
  • Added tests/TESTING.md — comprehensive testing guide for unit and E2E tests
  • Removed obsolete SLM-STRATEGY.md (7B models made the semantic translation layer unnecessary)
  • Fixed branch strategy inconsistency in CONTRIBUTING.md

Model Comparison (E2E Results)

Category Qwen 2.5 7B Hermes 2 Pro 7B
Single Tool Selection (5) 100% 80%
Conversational (3) 100% 100%
Multi-Tool Chains (3) 67% 67%
Error Recovery (2) 100% 100%
JSON Reliability (8) 100% 63%
Goal Completion (3) 100% 67%
Complex Multi-Step (2) 100% 50%
Conditional Logic (1) 100% 100%
Total (27) 96% 74%

Full Changelog: v0.2.0...v0.3.0