Skip to content

v6.5.0

Choose a tag to compare

@Shelex Shelex released this 09 May 17:54
· 75 commits to main since this release

Features

  • feat(llm): multimodal — attach failure screenshots to test analysis with the same auto-fallback blocklist; mode setting in Settings UI
  • feat(llm): customizable prompts via Settings (system + per-task instructions) with {{var}} mustache substitution from per-template allowlists; defaults themselves use the placeholder syntax so they double as canonical examples
  • feat(llm): segmented prompt builder with provider-aware cache_control (Anthropic) and KV-cache-friendly ordering (OpenAI / LM Studio)
  • feat(llm): structured output via Anthropic tool use / OpenAI response_format: json_schema with auto / force / disabled mode and 1h auto-fallback blocklist on unsupported models
  • feat(llm): context-window detection (Anthropic family registry, OpenAI /models probe with LM Studio / vLLM field aliases) plus manual override
  • feat(llm): token-count preflight (Anthropic exact via count_tokens, others 4-chars-per-token estimator) and budget-aware prompt fitting that drops cross-project context, attachment names, and history first; middle-truncates stack/error blocks last
  • feat(llm): per-task temperature settings (test / report / project), each with server-side defaults shown as input placeholders
  • feat(llm): max output tokens + context window override exposed in Settings UI
  • feat(llm): tiered cross-project context ranking (signature match dominates, recency decay tiebreaker)
  • feat(api): GET /api/llm/available-models with 5-min cache, powers Settings "Refresh models" button
  • feat(ui): LLM Queue page — Model column, Tokens (in/out) column, Test cell deep-links to served Playwright report
  • feat(ui): LLM Queue page — usage stats card with 7d/30d toggle (tasks, input/output tokens, reuse rate, per-type breakdown)
  • feat(ui): Settings — max output tokens, context window, structured output mode, multimodal mode, four custom-prompt textareas with "View default" disclosure + "Insert as override" button, "Refresh available models" with click-to-fill chips, per-task temperature inputs with default-as-placeholder
  • feat: properly persist config and branding images

Fixes

  • fix(llm): skip reusing older analysis when retrying
  • fix(llm): show proper model used for analysis retry
  • fix(llm): reuse-via-error-signature short-circuit with TTL (7d) and recurrence guard (5x) — skips reuse when environments may have drifted or the failure is no longer transient
  • fix(ui): LLM analysis injection survives retry-tab switches via body-level MutationObserver guardian; resolves the load race that required a page refresh
  • fix(llm): per-task temperature now actually honored — the queue was previously hardcoding 0.2/0.3 regardless of user settings
  • fix(ui): the inline LLM analysis renders even when the errors section isn't in the DOM yet on initial load (whenErrorsSectionReady waits up to 2s)
  • fix(ui): update the report page when analysis finished
  • fix(ui): content flash when retrying report-level analysis

Chores

  • chore(llm): SSE /api/llm/task-progress/:id replaces 3s client polling for queued analyses
  • chore(llm): drop LLM_TEMPERATURE env var and the runtime "default temperature" knob in favor of explicit per-task settings
  • chore(llm): drop token streaming due to structured output specifics, shiny text streaming does not justify complexity
  • chore(ci): update pnpm requirement, remove obsolete lockfile frozen arg, update node