Skip to content

v2.5 — Deployment Strategy

Choose a tag to compare

View all tags
@fhdmrddev-dotcom fhdmrddev-dotcom released this 09 May 18:32
v2.5
622d3ad

v2.5 — Deployment Strategy

Shipped: 2026-05-09 (10 days, 445 commits, 16 phases, 64 plans, ~104K LOC delta across 531 files)

Run-backed streaming + adaptive timeouts + UX polish. Agent generation lifetime is fully decoupled from any single HTTP request, multi-tab sync / refresh-mid-stream / navigate-away survive without manual refresh, complex tool-calling agents no longer cut off mid-iteration, and the skills test foundation is green for the next milestone.

What's new

Backend SSE concurrency unblocked (Phase 058)

Shared aexec(query) async wrapper around supabase .execute() plus AnyIO 200-token default thread limiter set in lifespan startup. Every Supabase .execute() reachable from event_stream (and the pre-stream user-message INSERT) is now wrapped in await aexec(...), plus 13 in-scope helpers and 6 cascade-forced kb.py helpers are now async def — the asyncio loop is no longer blocked by sync DB round-trips on the SSE path. Cross-tab GET drops from ~30s queued to <1s during streaming (CONCUR-01 binding pytest gate, measured ~15ms on green).

Run-backed streaming architecture (Phases 059 → 063 + 063.1)

  • Phase 059 — asyncio.Queue + agent_runner producer + EventSourceResponse(ping=15) replaces in-handler async generator and custom SSEStreamingResponse. Single task.cancel() signal with shielded partial-response persist.
  • Phase 061 — Redis Streams run:{run_id} durable buffer; producer XADDs, consumer two-mode XREADs from the same stream. Producer lifetime fully decoupled from the SSE consumer (D-061-03 contract inversion). New public.runs Postgres table for durable lifecycle counterpart.
  • Phase 062 — GET /threads/{tid}/active-runs and GET /runs/{rid}/stream?since={offset} HTTP endpoints. Replay-then-tail across all three sub-paths (live, already-terminal, TTL-expired). Cross-user 404 + Redis-down 503 best-effort degradation. DELETE /runs/{rid} server-side cancel.
  • Phase 063 — POST /threads/{tid}/messages returns HTTP 201 JSON {message_id, run_id} synchronously; legacy SSE-on-POST generator deleted. Frontend streamMessage orchestrator split into postMessage + subscribeToRun + getActiveRuns + cancelRun. useMessages rewired with reconcile(threadId) + resumeFromFailed(message) for mount/visibilitychange/focus/pageshow recovery.
  • Phase 063.1 — Closed 5 gaps from 063's live UAT (duplicate-bubble flicker, blank-window-on-thread-switch, hard-coded since="0", concurrent-reconcile race, Resume-button data flow).

Multi-tab sync, refresh-mid-stream, and navigate-away all work as a side-effect.

Adaptive run timeouts + lifecycle states (Phase 066)

Per-LLM-call budget that resets on tool-call boundaries replaces the 120s total-deadline. The agent loop has no hard total cap (matches ChatGPT/Claude UX); only individual LLM calls have per-call budgets. When a budget IS exceeded, the run terminates as timed_out (distinct from user-initiated cancelled), captures a clear runs.error = "timed_out after Ns at iteration N" row, and the UI surfaces an "Agent reached time limit" banner with a Resume button. Closes Gap-006, verified by a live 9-minute multi-step run.

Streaming UX polish (Phases 067 + 067.1)

  • Phase 067 — UX-067-01..05 closed: empty-paint, "Saving response…" thrash, refresh-required first-paint, redis-consumer log noise on tab cycle, tool-call iteration boundary surfacing.
  • Phase 067.1 — Track A drain-into-queue helper neutralizes langsmith trace noise on per-call timeout (closes Gap-007 / D-066-11). Context-aware in-flight copy ("Searching knowledge base…", "Setting up agent…", "Spinning up sandbox…") driven by iteration_start + tool_preparing event payloads. Multi-step-intent system prompt redesign for clear pipeline execution. Skill-load tool-card copy clarity.

Streaming render & storage fixes (Phases 067.2 → 067.5, cross-phase chain)

  • Per-thread message store — messagesByThread: Map<string, Message[]> (LRU N=5) replaces single-buffer pattern. Cross-thread switch mid-stream now preserves the streamed-into thread's render.
  • Sandbox download via JS blob fetch — frontend/src/lib/api.ts:downloadSandboxOutput follows 302 redirect, downloads as Blob, triggers programmatic <a download> click. No more 401 on browser anchor click.
  • Model→provider router — threads.py:949 honors MODEL_CAPABILITIES[model]['provider'] over user_settings.active_provider. Anthropic models actually route through the Anthropic SDK; Claude / OpenRouter / Google models work end-to-end.
  • Suggestions emit fix — Always-emit-empty + reordered before done so SSE-replay readers can actually receive it. max_completion_tokens=2000 + run_in_threadpool wrap.
  • Code-execution heartbeat — code_executing SSE event ~every 1s during sandbox execution carrying tool_index + elapsed_seconds. Frontend renders a live elapsed counter next to the existing spinner.
  • Empty-thread-until-refresh fix (Phase 067.5 Branch D-3) — clearMessages() now refuses to wipe a bucket whose thread is still streaming. 5/5 lived-experience cycles GREEN at user-driven UAT.

Skills test infrastructure repair (Phase 065)

  • 11 patch sites renamed from create_streaming_chat → create_adaptive_streaming_chat + 19 fakes tuple-wrapped to match the new (stream, calling_mode) return contract (065-01).
  • 3 export-test assertion drifts fixed against slug-prefixed bundle layout (065-02).
  • 11 tests migrated to canonical Phase 063 POST→GET-stream pattern using _build_mock_supabase() + per-test thread_id fixture (065-03).
  • Combined skills test run: 26/26 pass. Foundation for Skill Studio milestone.

Architectural decisions locked

Decision What it locks
D-v2.5-01 Blocking I/O in async handlers must be wrapped via run_in_threadpool / aexec
D-v2.5-02 Single uvicorn worker (multi-worker masks concurrency bugs)
D-v2.5-03 Realtime is best-effort hint, not source of truth — always reconcile via fetch on (re)connect
D-v2.5-08/09/10 STREAM-04 run-backed streaming architecture (Redis Streams + replay-and-tail)
D-v2.5-11 061 + 062 + 063 ship as a single feature branch — no feature flags, no dual code paths
D-066-11 stream.close() invariant under synthetic-timeout
D-067.3-N01 Model→provider router resolution chain

Stack changes

  • New runtime dependency: Redis (via docker-compose.dev.yml at repo root) — durable per-run stream buffer; configured to auto-start on Docker Desktop boot.
  • Local-vs-cloud switch: env vars only. SUPABASE_URL + REDIS_URL in backend/.env point at local containers by default; switch to cloud (Supabase project URL, Upstash rediss://...) without code changes.
  • New tables: public.runs (run lifecycle); runs:active + runs_by_thread:{tid} Redis sorted sets.

Carry-forward seeds (NOT v2.5 scope)

  • SEED-009 — claude-haiku-4-5 max_tokens=65536 > 64000 cap mismatch (surfaced during 067.5 cycle 5; Resume worked; needs per-model cap awareness)
  • SEED-010 — NR-067.2-11/12 OpenRouter Kimi 2.5 + MiniMax 2.7 synthetic-timeout protocol (creds present; ~30 min UAT to verify timeout machinery on OpenRouter-routed models)
  • SEED-011 — test_059_disconnect.py::test_normal_stream_unchanged Event-loop-closed fixture-teardown bug (pre-existing on fa1e327 base; needs _reset_redis_singleton autouse fixture analog)

Forward-looking strategic seeds

  • SEED-012 — Admin / Operator UI Completeness (control everything from UI; sane fallback when not)
  • SEED-013 — External Integrations (public REST API + MCP server + webhooks + service accounts)
  • SEED-014 — Automations & Routines (scheduled / triggered / reactive agent runs built on the existing skills + run-backed streaming substrate)

Deferred items

31 items acknowledged at close (10 UAT status fields not flipped after cross-phase closure — cosmetic only, all show 0 pending scenarios; 4 verification gaps marked human_needed — project-level approved per Phase 063 precedent; 11 historical micro-tickets predating GSD; 6 dormant seeds intentional future work). See .planning/STATE.md ## Deferred Items for the full inventory.

Phase 064 (Validation Harness) is intentionally deferred — scenarios E + F + H were validated organically by Phases 067.3 / 067.4 / 067.5 user-driven UAT (5/5 GREEN cycles for empty-thread-until-refresh + cross-thread switch).

See also

🤖 Generated with Claude Code

Assets 2
Loading