Background Conversation Generation

Background Conversation Generation

ChatHub generates chat replies on the server so closing a tab, navigating away, or restarting a replica does not cancel in-flight work.

The first message on a new topic shows the user bubble and assistant placeholder immediately, on a stable topic, without rebuilding the thread after the send is saved.

If that first answer still looks idle after the server has finished (later questions on the same topic stream normally), turn on CHATHUB_GENERATION_DEBUG=1 and send one new topic. Look for durable_attach and event_applied_terminal. Match them by operationHash (the attach line's hashed operationId), not by spanId. A later refresh can also show the finished answer without a terminal SSE event.

Clicking Regenerate again while a reply is still running must keep streaming the latest turn. The earlier job is cancelled. If the new bubble freezes after the server finished, search CHATHUB_GENERATION_DEBUG=1 for event_applied_terminal on that turn's operationHash. A missing terminal on the latest hash (not the cancelled one) means the tab dropped the finished done frame.

After a container recreate or PWA reload, a finished reply must not stay on Still working…. A healthy reconnect live-tails: sse_closed.deliveredCount stays small, and the bootstrap tail announces a reset at the latest cursor so ChatHub can resync the attached job. A reply that finished before that boundary can appear from the persisted messages without replaying its terminal SSE event. Thousands of delivered frames plus a missing event_applied_terminal was the old full-history SSE flood.

The feature is on by default. Disable it with:

environment:
  - FEATURE_FLAGS=-durable_conversation_generation

Only Stop, retry, rewind, delete, and clear cancel matching work. Closing the browser or disconnecting SSE leaves the job running. Portal Stop stays scoped to that thread and does not cancel a main-topic title, translation, or compaction job.

What you need

A server-reachable provider key or base URL (container env or Settings vault). Keys that exist only as “fetch in the browser” fall back to the connected-tab runtime for that send: the user message is saved, then ChatHub generates in the browser like before.

What continues in the background

• Chat, continue, regenerate, and group supervisor / group agent turns
• Web browsing, fixed Memory tools, HTTP MCP (including OAuth tokens already in Postgres), and the skills loader
• Artifacts (lobe-artifacts). This is a prompt-only builtin: the worker injects the Artifacts system prompt and the UI renders <lobeArtifact> tags. No browser runtime is required.
• Sandbox (lobe-sandbox) on OpenSandbox (Sandbox (OpenSandbox)). Enabling it does not defer the turn. When the model calls a Sandbox API, Graphile runs it and continues with the tool result (including a configuration error when the server is unset, or a program error). Stop during the call cancels the turn instead of handing that cancel to the model as a failed tool result. A portal-thread run uses that thread’s files (plus the main prefix), not sibling threads.
• Topic titles, translation, and guarded background memory compaction

Topic titles are guaranteed: if a title job starts before the new messages are visible on the topic, ChatHub retries it automatically (with backoff) until the transcript is available, so a topic caught in that race no longer stays untitled. If the title still cannot be generated after the final retry, only the title job is marked failed — the chat reply itself is never affected.

Background memory compaction is the opposite of that title-transcript race: an empty summary (for example a thinking model that spent the completion budget on hidden reasoning) fails that job once. ChatHub does not run Graphile's default eight retries on the same empty content. Chat history is unchanged; you can compact again after switching History Compress to a non-thinking model or sending more settled turns.

When a provider rejects a background turn, the retry status and the final error show the provider and its message (for example anthropic: messages.1.content.0: Invalid signature in thinking block…), and the server log prints the same text on each failed attempt.

After a finished reply, the message footer can show total time next to the token count (any model). Click the time and token figures on the right for TTFT, response time, and tok/s when the stream reported them. The model name on the left is not a trigger. Older messages without those times stay tokens-only.

What stays on the connected browser

• Image-generation chat tools (DALL·E and similar), non-HTTP MCP, and unknown plugins. ChatHub does not fake a server result for those; it runs them in the tab instead.
• The Image workspace already has its own async_tasks background path. That is unrelated to this conversation worker.
• TTS on the worker is metadata-only. Spoken audio persist stays client-side.
• Context Export records the browser request. When Export next request is armed, that send skips durable enqueue so the capture still works.

Leaving a topic is not Stop

Switching to another topic or chat/session on PC, tapping Back on mobile (inbox), or hiding the tab does not call Stop. A server job keeps running. When a turn stays on the browser (Kagi, DALL·E chat tools, non-HTTP MCP), that same tab also keeps going after you switch away — tools finish and the model summary starts without you coming back first. Closing the PWA or killing the tab still stops a browser-only turn. When you return, ChatHub reattaches a server job or refreshes the finished messages. You should see streamed text, not a stuck loading ring.

While the next model call is running, the empty assistant shows Planning next step (then Still working… after about 20 seconds). That line means the job is still attached. It is not Stop. Switching topic or coming back to the tab does not restart that 20-second clock. One send can keep calling tools for 32 rounds. Several tools in the same reply count as one round. Repeating the exact same tool call four times in a row stops that loop. A revised script or a different query keeps going. When either limit hits, the next message says what finished, what is unfinished, and the next step. Reached the tool-round limit appears only when the 32-round ceiling is used up, on the tool bubble, not on that written outcome. A short “I’ll continue” reply with no tool call is a finished answer, not a planning state.

Background continuation requires a server job. These plugins cannot run on Graphile and force the connected tab for the whole turn:

• Kagi search (kagi)
• DALL·E and similar image-generation chat tools
• Non-HTTP MCP

Enabling Sandbox does not force that fallback. OpenSandbox runs the call even after you close the tab. Closing the tab mid-turn still interrupts a leftover browser stream (Safari may surface this as Load failed) for tools that really need the tab (DALL·E, Kagi, non-HTTP MCP). ChatHub treats that as an interrupt: partial text is kept, and an empty ... placeholder is removed instead of becoming a pink unknown-error box. Do not retry that stream just because the tab became visible again.

When a send does stay on the tab, the result is never silently lost:

• A browser reply that finishes while you are looking at another topic is saved anyway; when you return, the answer is there.
• Chat Image tool cards keep generating after you switch topics in the same tab: tiles attach to the original topic. If you leave during Deep Thinking on iPhone/Safari, WebKit may abort the stream (Load failed) before those cards appear — stay until Prompt cards show, or send again. Stop is different from leave: a Stop after a task id is saved does not auto- start that unpaid create when you reopen the topic, including on another device and when the conversation save is interrupted. The cancelled placeholder is written as soon as you press Stop, without waiting for the rest of the conversation to finish cancelling. If the server never received Stop and this browser could not store the stop id, reopen can still recover as a crash; press Retry only if you still want the image. A later image started after Stop that crashed before the provider request still resumes on reload.
• If that reply asked for a tool or a Knowledge Base / RAG search, the same tab keeps going after you switch topics or sessions: HTTP MCP, built-in tools (web search, Sandbox, and similar), and KB retrieval finish, then the model writes the summary immediately, while you are still looking at the other chat. You do not have to come back first, and you should not need to type ?. The topic-list star keeps spinning until that follow-up finishes. Closing the tab still interrupts a browser-fallback turn — the tab is the producer. On mobile, tapping the topic name in the header opens the topic list; it does not leave a floating black title in the middle of the screen.
• If you come back before that continue started (or if it was skipped), ChatHub still resumes the model on return as a backup. A finished Tavily JSON with no next assistant message is that leftover case, not the healthy path.
• If you press Stop (or the tab dies) after some text already arrived, the partial text is kept. If nothing arrived, the empty ... placeholder is removed instead of staying as a dead bubble — including on those browser fallback turns.
• Should an old stuck ... bubble remain from a previous version, it is cleaned up automatically the next time you open that topic.
• While a topic is still generating (Deep Thinking, streaming, a tool call, or a leftover ... placeholder), its star icon in the topic list spins. That includes the topic you are looking at and a topic that kept going after you switched away. The icon returns to a star when that turn finishes.

Multi-step tool loops in the browser still need the tab to stay open after you return; remove the plugin from the assistant if you need a fully server-side turn.

Online Search (lobe-web-browsing), Artifacts (lobe-artifacts), and HTTP MCP (including Microsoft Learn) are allowed on the worker.

Threads and RAG

Main-topic and portal threads are separate lanes. History sent to the model matches the visible thread: main messages only, the source prefix plus a continuation thread, or the source message plus a standalone thread. RAG can still be injected via the chat turn’s retrieval query; it uses the dedicated embedding provider.

Group chats keep supervisor thinking on its own lane. Each member turn uses a per-agent lane so parallel or sequential replies do not cancel one another. Productive todos, response order, and follow-up supervisor rounds continue in the background. A later group message, regenerate, translation, or title request starts a new background job; retrying the same send after a lost response still attaches the original job. Stop, a failed supervisor round, cancelling a job that is still queued, or stale recovery clears leftover ... member placeholders. A member that already produced a reply is not marked failed when a sibling fails. Creating a member or tool-continuation placeholder and recording its id on the job happen in one database transaction. Parallel member ids are appended with a row-locked JSON update so two children finishing at once cannot drop one id. If a crash left the job pointing at a missing assistant row, restart recreates that ... placeholder and refuses to mark the job succeeded without it. Errors after a tool continuation land on the newest assistant, not the completed tool-call row. Stale recovery also pages through every finished job that has not yet been marked cleaned, so leftover ... rows are cleared even when more than 100 jobs finished. If one leftover cannot be cleaned, that job is skipped and later jobs in the same recovery pass are still processed.

Worker and database

The Next.js process starts a Graphile Worker against PostgreSQL. Job payloads are { operationId, userId } only; API keys are resolved at execute time from encrypted vaults and env.

Variable	Purpose
FEATURE_FLAGS=-durable_conversation_generation	Disable the feature
CONVERSATION_WORKER_CONCURRENCY	Graphile worker concurrency
DISABLE_CONVERSATION_WORKER=1	Do not start the worker in this process

Migrations 0054–0057 create conversation_generation_operations, events, and steps. Graphile owns the graphile_worker schema separately. JSON account backup does not include those tables or graphile_worker.

If replies hang after a crash, wait for stale recovery (about 90 seconds) or retry the message. Finished jobs that still need placeholder cleanup are paged until they are marked cleaned, so leftover loading placeholders do not stay on screen.

If the container never becomes ready and logs Cannot find module 'tslib' while loading graphile-worker, upgrade the image. That import happens while Next loads instrumentation, so disabling the worker env var does not avoid it.

Diagnostics

Background generation runs in the same Node process as HTTP chat. Recreate the container after changing these switches; no image rebuild is required.

Switch	Background path
CHATHUB_DEBUG=1 / LOG_LEVEL	Pino for tRPC and other server logs. Worker lifecycle uses [conversation-generation] console lines.
CHATHUB_TOOLS_DEBUG=1 / verbose	HTTP MCP still emits call_tool_* from the MCP service. Worker tool turns also emit tool_batch_started, tool_completion_reported, and tool_batch_settled.
DEBUG_<PROVIDER>_CACHE=1	Worker runtime.chat now attaches the same cache diagnostic context as /webapi/chat. Requires KEY_VAULTS_SECRET or NEXT_AUTH_SECRET.
CHATHUB_KNOWLEDGE_DEBUG=1 / verbose	Worker RAG inject emits retrieval / vector-search / prompt-injection records. Embedding still logs from RagEmbeddingService.
CHATHUB_IMAGE_DEBUG=1 / verbose	Image workspace (/image) async_tasks only. Chat Image tool (lobe-image-designer) tile create/persist/attach is not this stream — use CHATHUB_GENERATION_DEBUG.
CHATHUB_GENERATION_DEBUG=1	Structured send/worker events (chathub-generation-debug). A search-only send with Sandbox enabled should enqueue_persisted, not enqueue_rejected / lobe-sandbox. When the model calls a Sandbox API, look for sandbox_run_settled (not browser_tool_stubbed). A leftover iOS tab stream death is fetch_stream_interrupted (errorKind=webkit_load_failed), not UnknownChatFetchError. Worker DALL·E stubs emit browser_tool_stubbed. Continue fetches emit exec_runtime_settled(kind=continue). send_rpc_settled.deferReason stays a readable label. Leaving a background topic and returning shows deferred_lane_left then deferred_lane_resumed on the same send span. After MCP/tools finish, tool_loop_continue means the model started the summary; tool_loop_continue_skipped names why it did not. Off-screen RAG, builtin search, persist, and chat Image tiles emit invalidate_preserved, rag_retrieve_settled, builtin_tool_settled, message_persist_skipped, and chat_image_* on that span (hashed ids and counts only). Empty-title retries appear as execute_retrying, not eight unstructured throws.
DEBUG_*_CHAT_COMPLETION=1	Provider factories read the env at call time, including worker chat. Unsafe: may include prompts.

See Debug and Diagnostics for the full catalog.

See also Docker Deployment and Upgrades, Memory and Context Compaction, MCP Management, and Context Export.