Layer managed chat runtime behind stable chat_ask facade

[lloydrichards]

What to build

Evolve the generated chat scaffolding so the base chat module exposes a stable chat_ask facade while optional modules can add run coordination and SQL-backed Effect chat persistence without forcing users to replace an earlier chat-server choice.

The core direction is to keep the simple client-facing ask contract stable, then layer richer runtime behavior underneath it. Each layer should add the minimal surface area needed for future expansion.

Design direction

• chat_ask is the base facade: message history in, ChatStreamPart stream out.
• ChatRuntime.ask is the server-side service boundary refined by optional modules.
• Effect Chat should remain the underlying session/history abstraction; do not rebuild chat history persistence unless a future app-specific metadata model requires it.
• chat-server remains the simple starting point.
• chat-managed-runtime should refine chat-server with ephemeral run coordination, not persistent chat storage.
• Managed runtime concerns are runId, event streams, interruption, short replay, and active run coordination where needed.
• chat-react-client should keep using chat_ask and remain unaware of run IDs, interruption, or persistence.
• chat-manager-react-client can opt into run-aware behavior later.
• db-sqlite should be a reusable database foundation, independent of chat.
• chat-persistence should hook Effect Chat.Persistence to SQL via Effect Persistence, with no chat metadata/list/get/delete model for now.

Helpful context

• Reference repo: https://github.com/lucas-barake/effect-ai-chat-example
• Local reference checkout: .reference/effect-ai-chat-example
• Reference files of interest:
  • .reference/effect-ai-chat-example/packages/domain/src/api/chat-rpc.ts
  • .reference/effect-ai-chat-example/packages/server/src/api/chat/chat-run-manager.ts
  • .reference/effect-ai-chat-example/packages/server/src/lib/workflow-run-coordinator.ts
  • .reference/effect-ai-chat-example/packages/server/src/lib/ai-models.ts
• Effect reference files of interest:
  • .reference/effect/packages/effect/src/unstable/ai/Chat.ts
  • .reference/effect/packages/effect/src/unstable/persistence/Persistence.ts
• Current stack-effect chat content:
  • packages/catalog/src/registry/content/chat.ts
  • packages/catalog/src/registry/content/ai.ts
  • packages/catalog/src/registry/content/client-chat.ts
  • packages/catalog/src/registry/modules/server.ts
  • packages/catalog/src/registry/modules/client.ts

Child issues

• Simplify ChatStreamPart and rename chat facade to chat_ask #140 - Simplify ChatStreamPart and rename chat facade to chat_ask
• Introduce ChatRuntime.ask service boundary #141 - Introduce ChatRuntime.ask service boundary
• Add minimal domain contracts for managed chat runs #142 - Add minimal domain contracts for managed chat runs
• Add in-memory chat-managed-runtime behind chat_ask #143 - Add in-memory chat-managed-runtime behind chat_ask
• Expose minimal managed chat run RPCs #144 - Expose minimal managed chat run RPCs
• Add optional chat-manager React client #145 - Add optional chat-manager React client
• Add reusable db-sqlite foundation module #146 - Add reusable db-sqlite foundation module
• Add SQL-backed Effect chat persistence wiring #147 - Add SQL-backed Effect chat persistence wiring
• Evaluate WorkflowEngine-backed chat runtime #148 - Evaluate WorkflowEngine-backed chat runtime

Dependency map

flowchart TD
  I140["#140 Simplify stream + chat_ask"] --> I141["#141 ChatRuntime.ask"]
  I141 --> I142["#142 Minimal managed run contracts"]
  I142 --> I143["#143 In-memory managed runtime"]
  I143 --> I144["#144 Minimal managed run RPCs"]
  I144 --> I145["#145 Manager React client"]
  I146["#146 db-sqlite foundation"] --> I147["#147 Effect Chat persistence over SQL"]
  I143 --> I148["#148 WorkflowEngine evaluation"]
  I147 --> I148

Loading

Deferred until a concrete product need exists

• Chat metadata tables: title, model, owner/user, created/updated timestamps.
• Chat list/get/delete RPCs.
• Thread/chat inbox UX.
• Durable active-run metadata.
• WorkflowEngine-backed runtime implementation.

Open questions

• Should managed chat_start accept only message history initially, or include a generic owner key for active-run guarding?
• How long should completed in-memory run streams be retained for reconnect/replay?
• Can the current Catalog contribution model replace/refine a generated service layer cleanly, or do we need a new contribution type?
• When a future caller needs persisted history, should it pass a stable persistence key through a separate method/RPC rather than expanding base chat_ask?