Skip to content

v1.7.96 — Phase 1: LLM-as-Judge + Task Classifier + Token Optimization

Choose a tag to compare

@Zafer83 Zafer83 released this 17 Apr 14:42
· 2 commits to main since this release

What's New in v1.7.96

Phase 1: Multi-Agent Architecture

LLM-as-Judge Quality Gate
A second LLM call independently reviews proposed code edits before writing them to disk. The judge checks for:

  • Syntax errors and missing imports
  • Hardcoded secrets or eval() usage
  • Incomplete task fulfilment
  • + markers or line numbers leaking into file content

If the judge rejects edits with sufficient confidence (≥ 0.7 by default), the agent retries once with the judge's issues as explicit constraints.

Task Complexity Classifier
Zero-token regex classifier routes every request to the right pipeline tier:

  • trivial — deterministic path, no LLM (mkdir/delete)
  • simple — lean 400-token prompt, no judge
  • complex — full 1400-token prompt + judge gate

Token Optimization

  • Simple tasks now use ~400 tokens system prompt instead of ~1400 (saves ~1000 tokens/request)
  • Claude: cache_control: ephemeral on system prompt for complex tasks (~90% savings on repeat calls)
  • Gemini: fixed systemInstruction field separation

Mixed-Provider Judge
Route judge calls to a stronger model (Claude Sonnet, Gemini) even when the main coder is a local Ollama model. Configure vertexAgent.judgeProvider.

Claude Streaming
Claude provider now streams token-by-token via Anthropic SSE format.

English System Prompts
All system prompts rewritten in English for better cross-model compatibility (user-facing explanations remain in German).

New Settings

Setting Default Description
vertexAgent.judgeEnabled true Enable/disable judge quality gate
vertexAgent.judgeProvider "same" Provider for judge calls (same, claude, gemini, openai, ollama)
vertexAgent.judgeApiKey — API key for judge provider (when different from main)
vertexAgent.judgeMinConfidence 0.7 Minimum confidence to trigger retry
vertexAgent.multiAgentForOllama false Allow judge when main provider is Ollama

Tests

  • tests/unit/task-classifier.test.js — 52 assertions
  • tests/unit/judge-gating.test.js — 37 assertions
  • tests/unit/bugfix-regression.test.js — 61 assertions (unchanged, all pass)

Total: 150 unit test assertions