Skip to content

feat: add deep survey example and worker shared state support#4159

Merged
balegas merged 3 commits intomainfrom
vbalegas/deep-survey-example
Apr 24, 2026
Merged

feat: add deep survey example and worker shared state support#4159
balegas merged 3 commits intomainfrom
vbalegas/deep-survey-example

Conversation

@balegas
Copy link
Copy Markdown
Contributor

@balegas balegas commented Apr 23, 2026

Summary

Port of electric-sql/durable-streams#134.

Adds a new Deep Survey example app (examples/deep-survey/) — a demo where ~50 AI agents explore a corpus in parallel, each doing real web research and writing wiki entries with cross-references. Also adds shared state DB support to the built-in DARIX worker entity type.

Root cause (bundler fix)

The db() observation source in ts-darix-runtime used property shorthand (schema,) which the tsdown bundler mangled to dbSchema: schema in the dist output. This caused client.observe(db(...)) to fail silently because the schema property was missing from the returned object. Fixed by renaming the parameter to dbSchema with explicit schema: dbSchema assignment.

Approach

Backend — Two entity types:

  • orchestrator: Receives a target corpus, does initial web research (Brave Search API), then spawns N worker entities each assigned a topic. Uses shared state to accumulate wiki entries and cross-references.
  • Workers use the built-in DARIX worker type (extended in this PR to support shared state DBs). Each worker gets brave_search + fetch_url tools plus auto-generated CRUD tools for the shared wiki/xrefs collections.

Built-in worker shared state support (ts-darix-server):

  • WorkerArgs now accepts optional sharedDb: { id, schema } and sharedDbToolMode: "full" | "write-only"
  • buildSharedStateTools() generates write_, read_, update_, delete_ tools per collection
  • parseWorkerArgs validates sharedDb structure and throws on malformed input
  • At least one of tools or sharedDb must be provided

Frontend — Real-time reactive UI:

  • useSwarm hook observes entities by tag + manifest-gated shared state subscription
  • Dark monospace theme, grid layout with chat sidebar, agent graph, wiki column, activity panels
  • createEffect from @tanstack/db for live collection updates

Key invariants

  • clone_repo tool validates URLs against ^https?://[a-zA-Z0-9._\-]+/[a-zA-Z0-9._\-/]+$ and uses execFile (no shell interpolation)
  • Workers are sandboxed — no ctx.darixTools, only the tool subset chosen by the spawner
  • Schema primary keys use z.string().min(1) to prevent empty-key insertion
  • Shared state manifest must appear in the orchestrator's stream before the frontend subscribes (prevents race with ctx.mkdb())

Non-goals

  • No schema validation on shared state writes in the built-in worker (would need Standard Schema integration — tracked separately)
  • No retry logic for transient shared state connection failures in the UI
  • as any casts in useSwarm.ts remain — fixing requires upstream type improvements to the darix-runtime client API

Running the example

Prerequisites

  • A running Electric Agents server (DARIX) — start one with the CLI quickstart:
# From repo root — starts Docker infra (Postgres, Electric, agents-server) + built-in Horton server
node packages/electric-ax/bin/electric-dev.mjs agent quickstart
  • ANTHROPIC_API_KEY set in .env at repo root (required for the agents)
  • Optional: BRAVE_SEARCH_API_KEY env var for web search (set in .env at repo root)

Setup & run

# 1. Install dependencies (from repo root)
pnpm install

# 2. Build the agents-runtime package (required — example uses workspace:* link)
pnpm --filter @electric-ax/agents-runtime build

# 3. Start the example server and UI in separate terminals
cd examples/deep-survey

# Terminal 1 — entity server (registers orchestrator with DARIX)
pnpm run dev:server

# Terminal 2 — Vite UI dev server
pnpm run dev:ui

Note: Do not use pnpm dev — it runs all dev:* scripts across the workspace. Use dev:server and dev:ui individually.

This starts:

  • Server on http://localhost:4700 — registers the orchestrator entity type with DARIX
  • UI on http://localhost:5175 — Vite dev server with proxy to the backend

Environment variables

Variable Default Description
DARIX_URL http://localhost:4437 DARIX server URL
PORT 4700 Backend server port
SERVE_URL http://localhost:$PORT Webhook callback URL for DARIX
BRAVE_SEARCH_API_KEY Brave Search API key for web research

Type-check

cd examples/deep-survey
pnpm typecheck

Files changed

New example app (examples/deep-survey/):

  • src/server/ — orchestrator entity, explorer prompt, schema, HTTP server
  • src/ui/ — React frontend with 7 components, useSwarm hook, dark theme CSS
  • Config files: package.json, tsconfig.json, vite.config.ts

🤖 Generated with Claude Code

@balegas balegas force-pushed the vbalegas/deep-survey-example branch from 1b53a99 to c7b7419 Compare April 23, 2026 17:44
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 23, 2026

Codecov Report

❌ Patch coverage is 26.56250% with 47 lines in your changes missing coverage. Please review.
✅ Project coverage is 67.91%. Comparing base (3cef0ca) to head (91b4d38).
⚠️ Report is 2 commits behind head on main.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
packages/agents/src/agents/worker.ts 22.95% 47 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4159      +/-   ##
==========================================
- Coverage   68.08%   67.91%   -0.18%     
==========================================
  Files         130      130              
  Lines       16211    16264      +53     
  Branches     3912     3920       +8     
==========================================
+ Hits        11037    11045       +8     
- Misses       5170     5215      +45     
  Partials        4        4              
Flag Coverage Δ
packages/agents 37.16% <22.95%> (-1.24%) ⬇️
packages/agents-runtime 81.34% <100.00%> (ø)
packages/agents-server 65.55% <ø> (ø)
packages/agents-server-ui 0.00% <ø> (ø)
packages/electric-ax 30.46% <ø> (ø)
packages/experimental 87.73% <ø> (ø)
packages/react-hooks 86.48% <ø> (ø)
packages/start 82.83% <ø> (ø)
packages/typescript-client 94.30% <ø> (ø)
packages/y-electric 56.05% <ø> (ø)
typescript 67.91% <26.56%> (-0.18%) ⬇️
unit-tests 67.91% <26.56%> (-0.18%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@balegas balegas changed the base branch from main to vbalegas/fix-example-builds April 23, 2026 18:30
@balegas balegas force-pushed the vbalegas/deep-survey-example branch from cc7640a to 3f1ba5c Compare April 23, 2026 18:30
Base automatically changed from vbalegas/fix-example-builds to main April 23, 2026 18:51
balegas and others added 2 commits April 23, 2026 20:01
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Port the deep survey (wiki-swarm) example from the durable-streams repo.
Add shared state DB support to worker agents, enabling spawners to give
workers per-collection CRUD tools via sharedDb spawn arg.

- Worker agents accept optional sharedDb and sharedDbToolMode spawn args
- Rename schema param to dbSchema in db() observation source
- Add deep survey example with orchestrator + explorer swarm pattern

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@balegas balegas force-pushed the vbalegas/deep-survey-example branch from 3f1ba5c to eea8c7b Compare April 23, 2026 19:02
…onflicts

Transitive deps pulled @types/react@19 alongside the direct @types/react@18,
causing dual csstype versions (3.1.3 vs 3.2.3) and widespread TS errors.

- linearlite, linearlite-read-only, todo-app: upgrade @types/react and
  @types/react-dom to v19
- linearlite: upgrade react-markdown 8→9 (v8 ships .ts files incompatible
  with React 19 types)
- Fix React 19 type changes: useRef args, RefObject<Element | null>
- Remove stale @types/react-router-dom from linearlite-read-only

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@balegas balegas force-pushed the vbalegas/deep-survey-example branch from eea8c7b to 91b4d38 Compare April 23, 2026 22:01
@balegas balegas merged commit 097f2c4 into main Apr 24, 2026
52 checks passed
@balegas balegas deleted the vbalegas/deep-survey-example branch April 24, 2026 11:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants