v2.8 — Harness Engine & Workflow Mode

Agentic RAG v2.8 — Harness Engine & Workflow Mode

v2.8 gave the agent a deterministic, auditable workflow runtime: locked, ordered phases the LLM cannot escape, dispatcher-enforced per-phase tool whitelists, and validation gates between steps — plus a per-thread Deep ↔ Harness dual-mode toggle and a live phase timeline in the workspace panel. It was built almost entirely by composing shipped v2.7 primitives (tool dispatcher, task/ask_user, run-backed Redis streaming) with zero new dependencies, and Deep Mode stayed the unchanged, byte-identical default throughout (the red line). This is the engine that v2.9 Workflow Studio is authored on top of.

Highlights

• Harness Engine — a hand-rolled, deterministic locked-workflow runtime with 5 phase types, validation gates, bounded retry, and step/wall-clock caps. ~80% composition of shipped substrate, zero new dependencies.
• Per-phase tool whitelist + tool budget — enforced at the single dispatch guard, a literal no-op when no workflow is active, so Deep Mode is byte-identical. Definitions are versioned + immutable-on-publish with an INSERT-only audit trail.
• Agent-loop + provider-gateway extraction — the agent loop was lifted out of the threads.py god file, then per-provider streaming was extracted into one shared provider_gateway/ that both Deep and the harness consume — the structural fix that brought the harness to cross-provider parity.
• Dual-mode wiring + cross-provider parity — a per-thread Deep/Harness toggle with a server-enforced workflow lock and a Continue affordance; the harness reached native-7 parity by consuming the gateway (live re-UAT 8/8).
• Legibility + run honesty — a live WCAG 2.1 AA phase timeline + RunCard in the panel; failed runs render as failed with a reason (no fake "done"); unified chat tool-cards; deterministic activity-derived panel fill; true reload timer; provider/model attribution.
• Eval gate + concurrency + resumability — a cross-provider eval harness wired as the CI regression gate (which caught a live security gap on its first run), restart-mid-workflow resumability verified at 3 kill points, bounded parallel fan-out, and an 8-provider model curation pass.

What's New in Detail

Harness Engine + phase types + gates (091)

harness_engine.run_workflow is a hand-rolled async transition loop that drives a published WorkflowDefinition through phase_index order, with a strict 2-phase write (mark-active → execute → atomic complete-with-output) and publish-time reachability lint. A PHASE_TYPE_REGISTRY dispatch seam is filled by 5 thin executors — programmatic, llm_single, llm_agent, llm_batch_agents, llm_human_input — that call the existing task_service / ask_user_service / tool_dispatcher, never re-implementing loops. Validation gates (4 kinds, closed registries) ride a bounded-retry loop (≤3, with consecutive-identical short-circuit) and on_failure routing (fail_run / retry / skip_to_phase), plus per-phase step and wall-clock caps.

Per-phase tool whitelist + immutable, audited definitions (091)

The per-phase tool whitelist + tool-count budget are enforced at the single dispatch_tool() guard and the get_tools() composition site — both literal no-ops when no workflow is active, so Deep Mode is byte-identical. Workflow definitions are versioned and immutable-on-publish (UNIQUE(slug, version) + a BEFORE-UPDATE trigger + FK ON DELETE RESTRICT), with FK-chain RLS and an INSERT-only harness_audit trail (migrations 056–062). A run-claim lease (CAS) means WORKER_COUNT=2 workers never double-execute a run.

Agent-loop + provider-gateway extraction (089 + 092.5)

The agent loop was lifted out of the threads.py god file into a clean agent_loop.py (proven byte-identical across the native-7), then its per-provider dispatch + chunk normalization was extracted into a shared provider_gateway/ package — canonical events, a dispatcher open_stream, three verbatim provider adapters, and one unified _on_chunk consumer — that Deep and the harness both consume. Surfacing calling_mode through this seam was the structural fix for the "harness only worked on OpenAI" bug. Deep was proven byte-identical in isolation: this is the red line.

Dual-mode wiring + native-7 parity (092 + 093)

A per-thread Deep/Harness toggle keyed on threads.active_workflow_run_id, a server-enforced workflow lock (HTTP 409 on an illegal switch, cleared in the terminal-status transaction), and the Continue affordance (resume past a step cap, consume-not-drop). The harness reached native-7 parity by consuming the gateway: a shared model-resolver (resolve, never mutate saved settings), a fixed ask_user round-trip, the three never-run phase types completed (split_topic → a real N-way llm_batch_agents fan-out), and provider-specific round-trips fixed for Google (thought_signature), Moonshot (reasoning_content), and GLM (max_steps). Live re-UAT: 8/8.

Legibility + run honesty (094 + 095 + 095.1)

A live, WCAG 2.1 AA phase timeline + a harness RunCard in the v2.7 workspace panel, demuxed into a dedicated phasesByThread store (zero chat re-renders). Failed and gate-failed runs render as failed with a reason — no done-sentinel lie. Chat tool-cards were unified into one frame (auto-scroll, details-on-demand, no duplicates, working downloads). And cross-provider run honesty: deterministic activity-derived panel fill (works even for providers that never call write_todos), 429-vs-billing classified at the gateway boundary on structured status codes, a true completed_at − started_at reload timer, model/provider attribution, and a deliverable-aware Resume gate.

Eval gate + concurrency + resumability (096)

scripts/eval_cross_provider.py was extended to drive multi-phase workflows per provider and wired as a CI regression gate — which caught and fixed a live phase_whitelist-not-propagated security gap on its very first run. Restart-mid-workflow resumability was verified live at 3 kill points (programmatic / llm_agent / ask_user) including graceful-shutdown; llm_batch_agents fan-out is bounded by max_parallel_agents composing with the global Redis-Lua cap; a StreamsProvider thread-keyed LRU-3 live-stream pool closed the thread-switch saturation hang; and an 8-provider, newest-first model curation pass landed.

By the numbers

• 10 phases (089–096, including the inserted provider-gateway refactor 092.5 and the inserted live-UAT phase 095.1) · 67 plans
• 9 days (2026-05-30 → 2026-06-07) · 498 commits (121 feat)
• 578 files changed (+107,662 / −6,187 lines)
• Migrations 056–066 (additive — harness tables, dual-mode, audit trail, eval seed)

Architectural decisions locked

• D-v2.8-01 — v2.8 is Harness Engine + dual-mode only; the 6-type Plugin Contract and the operator/admin role tier defer to a later milestone (lock them on real harness telemetry).
• GATEWAY-01 — one shared provider gateway is the single home for all provider logic; the harness reaches parity by consuming it, never re-implementing. Deep byte-identical is the red line.
• PARITY-02 over PARITY-01 — the parity the milestone needed was the harness path, not Deep-mode Anthropic polish (Deep is already provider-robust on all 7).
• D-094-UNIFY — the workspace panel is the single live-execution surface for both Deep and Harness.
• D-095.1 — run honesty is projection/classification over data that already exists (no migration, no new SSE event); provider-specific handling stays at the gateway boundary, never the shared path.

Quality & verification

Milestone audit (tech_debt): 24/25 requirements satisfied + 1 partial (CONC-01 → SEED-065-B), 10/10 phases verified & closed, 6/6 E2E flows. Every streaming / agent-loop / provider-routing / UI-state phase carried the SC#10 4-axis cross-provider UAT (cross-provider × multi-tool × parallel-thread × long-message). Full audit: milestones/v2.8-MILESTONE-AUDIT.md.

Deferred / known at close

43 items acknowledged (operator-approved accept-as-tech-debt). Headline: CONC-01 partial → SEED-065-B (cross-tab GET p95 ≈ 2,958 ms, ~2.9× better; residual ~3 s from a synchronous stream-create). Plus dormant forward seeds (SEED-002/003/004/005/040–046), pre-GSD micro-tickets, and the carried-forward Plugin Contract, llm_judge, and visual builder — all of which became v2.9 Workflow Studio.

Upgrade & migration notes

• No breaking changes to Deep Mode — it stayed byte-identical; the harness is additive (new tables, new optional fields, no-op guards when no workflow is active).
• New migrations 056–066 apply additively (new harness tables + audit trail + dual-mode columns + eval seed).
• This release introduced no new runtime and no new dependencies — it composes the shipped agent loop, tool dispatcher, and run-backed streaming.

Full changelog

See CHANGELOG.md for the complete history, and .planning/MILESTONES.md / milestones/v2.8-ROADMAP.md for the full close-out record.