Chat Pipeline

Chat Pipeline

End-to-end path for a chat turn inside the desktop app.

Entry

Renderer (chat.ts) calls into main via preload. Start is typically an IPC send (chat:start) with a payload including:

• Conversation / project / thread ids
• Model name
• Messages
• Attachments
• useTools
• skillOverrides (per-conversation skill enablement, e.g. URL chip → web research)

Responses stream back on channels such as chat:chunk, chat:activity, chat:governance, tool confirm / ask-user, file cards, and completion/error events. Exact channel names live in preload.ts + ipc.ts + api.d.ts.

Stages in main (ipc.ts)

1. Governance preflight

governance / policy engine:

• Org/user limits (cooperative)
• Model allowlists
• Content / PII evaluation
• Warnings pushed to UI without always hard-blocking (policy-dependent)

2. System prompt assembly

Typical ingredients (order can vary; read ipc.ts for the source of truth):

• Project instructions
• Retrieved RAG chunks (context.retrieve) or reference summaries
• Cross-thread memory (memory.recall)
• Optional general/org vectorstore hits
• Attachment text
• Skills instructionsBlock
• Workspace / project-coding block
• User Customize context (user-context.ts)
• Document-intent nudges when relevant

3. Routing: single vs multi-agent

If agents are enabled in settings and complexity classification says so → multi-agent path (agents/*).

Exceptions:

• Project-coding intent forces single-agent for that turn.
• Soft-fail planning carefully — comments in ipc.ts warn against falling back into single-agent after tools already ran (side effects).

4a. Single-agent tool loop

1. Build OllamaToolDef[] from tools registry + skills registry (if tools on).
2. ollama.chatStream with messages + tools.
3. Forward tokens / reasoning to renderer.
4. On tool calls: confirm if risky → execute → append tool results → loop.
5. Stop when no tool calls or rounds >= 15.
6. Recovery: parse pasted JSON tool calls from weaker models when needed.

4b. Multi-agent path

1. Classify / plan (agents/plan, classify).
2. Run workers with clamped concurrency and 3 tool rounds each.
3. Synthesize final answer.
4. Emit agent trail activity for the UI (collapsed by default).
5. Load-guard can soft-stop when RAM crosses threshold.

5. Persist & remember

• Save assistant/tool messages to project thread or standalone chat store.
• Embed/remember into Chroma memory collections.
• Knowledge-graph extraction (graph/).

Renderer responsibilities

• Maintain conversation UI state (state.ts)
• Render streaming tokens and activity strip
• Show tool confirm / ask_user modals and reply via IPC
• Handle file/artifact cards
• Respect Tools toggle and skill chips

Related knobs

Knob	Location
Tools default per project	Project settings / model lock specs
Agents enable + limits	Settings → Agents → llmeter-settings.json
Load protection	Settings + load-guard/
Embed model	ANYLM_EMBED_MODEL
Proxy (separate path)	Does not use this pipeline’s tools

Debugging tips

1. Confirm Ollama tool support for the selected model.
2. Watch activity trail for tool names and errors (Error: … strings from exec).
3. If RAG empty: check Chroma startup logs / embedding model pulled.
4. If skills “do nothing”: skill enabled? Tools toggle on? Connector connected?
5. Deny confirm once — ensure auto-deny timing still feels right for your change.