Memory and Context Compaction

Memory & Context Compaction

ChatHub separates topic-level compaction from assistant-level memory.

Assistant → Memory

Open Assistant (pick the agent) → Memory. The page has three areas: the Enable assistant memory master switch, Fixed memory (user-curated), and the Dreaming Memory group (topic snippets, dream schedule, and the dynamic-memory editor). The Compact group (Assist preset, auto-compact, threshold) lives under Assistant → Chat Preference, not here.

Each section title and Topic snippets carry inline ? help icons (hover or click). There is no separate setup banner.

The Enable assistant memory switch is the master control for that assistant. When it is off, ChatHub does not inject fixed or dynamic memory, run memory rollups, or offer the model's memory tools. The remaining compaction settings still work independently.

Fixed memory (user-curated)

User-curated facts, preferences, and standing instructions that remain available across every chat with this assistant. The title-row help icon explains that fixed memory is yours — the assistant can append entries via its built-in memory tool, and automatic summarization and the memory dream never rewrite it.

• Use Add memory to create one entry, or edit and delete individual cards. Entries are numbered automatically and renumbered after deletion.
• Use Edit for the raw document view when you need free-form Markdown. Saving that view also normalizes numbered entries without changing free-form lines.
• In a normal one-to-one chat, the model can add, correct, or remove fixed-memory entries when appropriate. These memory actions are hidden from the plugin picker and are not offered for group/member requests.
• Ask the assistant explicitly to forget something when you want it removed. The model verifies the current entry before an update or deletion so a stale number cannot overwrite the wrong memory.
• The model can also search and read its memory on demand (searchMemory / readMemory memory actions). You will not normally notice this — it matters when memory grows very large: the injected memory block is then capped to a budgeted head portion with a truncation marker, and the model uses search/read to recall the rest instead of guessing. Search results show a short snippet centered on the matched text (with a leading … when the match sits deep in an entry), and the model can then read that one entry by its number instead of re-reading the whole document. Long entries are read in pages so one tool result does not dump the whole entry: when a page comes back truncated, the model follows the returned continuation offset until the entry is complete, so even very large entries remain fully recallable.
• Trusted vs. untrusted memory — ChatHub tracks where each memory entry came from: entries you write in this editor are owner, dream cards are dream, and entries the model writes with its memory tool are agent. If the model writes a memory entry while its recent chat history contains output from external tools (MCP plugins or web browsing), the entry is downgraded to untrusted — a guard against prompt injection persisting itself into memory. This check runs in both background (durable) turns and in-tab generation, including when you switched topic after sending. Untrusted entries are still injected, but in a separate clearly marked section that tells the model to treat them as data, not instructions. Everything renders exactly as before when no untrusted entries exist.

Dreaming Memory

Stored-summary behavior, the scheduled memory dream, and dynamic memory. The section title help icon summarizes how topic snippets and Chat Preference compaction feed the dream and where output lands.

• Topic snippets — When on, stores short snippet text on each topic so the model can reuse prior summaries. The help icon notes that the dream prefers compaction summaries (which need Limit history messages and Summarize history on Chat Preference); a topic without a summary still joins the dream through a recent-message excerpt.
• Daily memory dream — Server-side schedule (Off / Daily / Weekly plus a UTC time, and a UTC weekday when Weekly). At the configured UTC time, ChatHub reads the topics that were active on the previous UTC calendar day — each topic's compaction summary, or a bounded excerpt of that topic's own recent messages from that day when it has no summary yet — and learns how you work with this assistant (tone, format, tool habits, standing preferences) into dynamic memory. It is not a recap and it does not compact topics. The browser does not need to be open. When the schedule is on, the group also shows the last dream run — the latest scheduled attempt's time, with a failure hint when that attempt failed (it retries on the next cycle). Only scheduled dream attempts appear here; a historical manual Regenerate from before this release is never shown as a dream run.
• Keep dream cards — How many single-day dream cards to keep before older ones are merged into one range card (default 14, range 1–90). Saving this form atomically updates the setting and applies retention in one server write.
• Dynamic memory — Shown at the bottom of this group. Durable facts extracted from topic compaction summaries or recent-message excerpts. Each successful dream run appends a new numbered card tagged with that run's UTC history day (YYYY-MM-DD), not a full rewrite of the whole document.
  • Cards work like fixed memory: #N, a date chip, and the body. Edit or delete individual cards. Regenerate (refresh icon) re-runs the dream for that card's UTC day only; it is hidden on merged range cards (YYYY-MM-DD – YYYY-MM-DD) and on legacy-wrapped blobs.
  • Copy exports the same visible card text as the editor (the internal overflow marker is not included); Clear removes all cards. There is no Add memory — only the scheduled dream creates dated cards.
  • When you keep more single-day cards than Keep dream cards, the oldest ones are re-summarized into one range-tagged card when the memory dream runs (same History Compress model). The model is told the new summary must be under 6400 characters; if the first draft is over that cap, or the provider stops because it hit a token limit, ChatHub asks it to rewrite a shorter complete summary instead of keeping a cut-off draft. If those calls fail, ChatHub concatenates as a fallback. Oldest folded days are dropped first so the newest overflow is kept. An overflow card that ChatHub cannot parse as dated sections stays as one opaque range: folding another day keeps its newest tail instead of deleting the whole range, even if that day's text looks like a [date:] or [day] heading. That special range card may be up to 6400 characters; each single-day card stays at the 3200-character limit. Each folded day in a concatenated fallback starts with a [date:YYYY-MM-DD] heading. A body line that is only [YYYY-MM-DD], only [overflow:v1], only [overflow:opaque-v1], or only [overflow:opaque-v2] stays ordinary text, including when it already has leading backslashes. A body line that is only [date:YYYY-MM-DD] is a day heading; if that exact line is ordinary text, the editor shows it as \[date:YYYY-MM-DD] so saving does not turn it into a heading. A leading [overflow:v1] is hidden only when it is ChatHub's real overflow framing for that card's date range; a literal copy of that line on an older card stays visible. A leading [overflow:opaque-v3 n=… crc=…] line is hidden framing only when its length and checksum match the rest of that range card. [overflow:opaque-v1], [overflow:opaque-v2], and a v3-shaped line that fails that check stay ordinary text even as the first line of a range card. The whole document, including card headers, stays within N × 3200 + 6400 characters.
  • The dream does not require topic summaries: a topic active yesterday that has no summary yet still feeds the dream through a bounded excerpt of its recent messages from that day. A run only finds nothing when no topic was active on the previous UTC day, or the active topics have no user/assistant messages yet. Enabling Limit history messages and Summarize history under Assistant → Chat Preference still helps — pre-digested summaries give the dream a cleaner signal than raw excerpts.
  • Near-duplicate cards are skipped: if a new dream card is essentially a paraphrase of an existing card (high token overlap, CJK-aware), the run commits its schedule marker but does not append, so repeat days do not pile up lookalike cards.

Current topic summary

Topic compaction is scoped to the topic rather than the assistant settings. Open its read-only viewer from the token-badge popover for the active topic, or from a summarized topic's menu. The viewer includes the summary, model tag, and copy/export action.

Token badge: next request vs topic

The chat-input token badge popover (Context usage & compaction) reports a next request estimate, not the size of everything visible in the topic. While you type, the draft token count updates after a short pause (~300ms); the window and topic-chat figures do not re-serialize the whole topic on every keystroke, so a long thread stays scrollable.

Scrolling a long topic keeps the same highlighted bubbles. ChatHub does not switch to a plain-text scroll view, because that swap jumped the thread when the wheel stopped. On the installed PWA and mobile chat, the topic sheet unmounts while it is closed. The session list stays mounted so Back is instant, but it does not keep every session row subscribed while you are inside a topic.

• Next request estimate / Total Used — what the next model call will carry (instructions, tools, memory, history summary wrapper, sliced chat messages, draft, images, and in-flight Knowledge Base). Text is counted with the same tokenizer for short and long messages. Each chat row also gets a small framing allowance. Images with a data URL are counted from their pixel size for known OpenAI and Claude models; other images and videos use a fixed allowance and are not counted as text. When a completed assistant reply in that sliced window carries a provider-reported totalInputTokens (the number on the message bubble), ChatHub anchors on that exact provider count and tokenizes only the messages after it (that reply's own text plus newer rows and your draft). A per-model correction, learned from earlier completed replies, scales only those locally counted tokens. That correction reloads when a completed reply reports usage, including usage stored only on the message metadata. Typing or streaming does not reload it. A reply whose skill text was not already available is skipped for that correction, and the send still goes out. A thread counts the earlier topic messages that request will send; if the thread's source message is missing, that reply is not used to learn the correction. An old report is not trusted on sight: after a page reload or in a new tab, ChatHub shows the larger of a fresh whole-window estimate and the report plus the new tail, so a stale report cannot shrink the badge and a paste after that reply is not ignored. The anchor stays bound to the request this tab actually sent: background (durable) sends freeze the assistant instructions and history window before the enqueue call, so editing those settings while that call is in flight does not rewrite the baseline. Changing skills or instructions while a reply is generating, or switching to another topic and back, does not get folded into that reply's anchor — the estimate keeps the pre-change baseline and adds the difference, or shows a fresh whole-window count if this tab never sent that request (reload / another tab). Raising or turning off the history limit — including raising it from 0 — or an automatic large-window expand that newly includes older turns — also drops the old report and recounts the full next request, so those older messages cannot vanish from the badge. Changing the input template after that send does the same: the template is applied to every included user message, so added prompt text on earlier turns is counted again instead of being omitted from the anchored tail. When no reply in the window has provider usage yet, ChatHub uses the whole-window tokenizer estimate. If a report exists but this tab did not send that request, the badge uses whichever is larger: that whole-window estimate, or the report plus tokens added after it. After Compact, that anchor ignores the protected assistant that was already billed (the last visible reply still carries the pre-compact provider total, which included turns that are now in the summary). Editing that reply, or a clock difference between the browser and the server, does not restore the old 1M anchor. Deleting the last remaining message after the summary cursor keeps the anchor at that cursor, so the next completed reply can become the anchor again. A new completed reply after Compact becomes the anchor again. The small Used N chip uses this number. Chat Messages stays roughly flat once the history window is full; that is expected.
• History window — how many topic messages are included vs dropped (by compaction cursor or history limit), the configured history limit (your saved setting), an optional large-context expanded to N suffix when the runtime effective cap exceeds that configured value, a topic chat estimate so growth is still visible, last compaction status, and a warning when older turns are excluded without a usable summary.
• Most recent reported cache — cached tokens divided by all provider-reported input from the latest assistant reply in this topic that included cache fields. Durable (background) turns persist those counters flat on the assistant message metadata, the same shape as a browser turn. Older rows that stored { usage: { … } } are still read. Tool-loop grouping, when applied, copies the same fields onto usage / children. A newer completed reply that reports only totals does not replace a previous cache-bearing rate. If no reply reported cache counters, the block shows 0% hit / provider reported against that reply's total input. The help icon explains that this is not the next-request estimate above it and names the model that reported the cache; the status line itself stays 99.2% hit · working · 18,432 / 18,579 without a from <model> suffix.

If no reply in the topic has reported cache counters yet, the block shows No provider telemetry yet. A first Anthropic write-only turn can show 0% hit / no hit yet while cache is being created. This popover rate is also separate from the OpenAI-compatible cache probe matrix and from per-bubble generation details.

Compact now and View topic summary still run topic compaction; they do not reset or recompute the reported cache rate.

Short Setup Path

Assistant → Memory → leave Enable assistant memory on → add any fixed facts under Fixed memory → adjust Dreaming Memory (topic snippets, dream schedule, Keep dream cards; edit Dynamic memory cards at the bottom of that group) → Save. The Compact group (Assist preset, auto-compact, threshold) is under Assistant → Chat Preference. For dynamic memory, also enable Limit history messages + Summarize history under Assistant → Chat Preference, create or compact some topic history, and let the memory dream append a new dated card at the scheduled time.

Topic Compaction (per topic)

• Assistant → Chat Preference includes history limits, auto summary, and the Compact group (Assist preset, token auto-compact with a configurable high watermark, default 0.8). The low watermark is always 0.2 lower. The memory dream is a separate server schedule and does not compact topics.
• Manual compact — the button in the token popover runs the same incremental summarization path. While it runs, Compact context now stays disabled and shows Compacting… until that run finishes, including background jobs, so a second click cannot start another compact. When background generation is on, it queues a server job and toasts Compacting context in the background (it does not mean the topic cannot compact). Other results: compacted, nothing eligible, failed, or could not reach the target. A tiny Covered by summary count (for example 4) with Total Used still near the previous value is expected: Compact only removes the turns it summarized from the next request; the rest of the history window stays on the wire. View topic summary can be available while Total Used is still high.
• Summarizer model — compaction uses Settings → System Agent → History Compress, not the model you are chatting with. The default is GPT-5 mini. The summary output cap scales with Assist preset (minimal ~400, balanced ~600, rich ~800 tokens). ChatHub gives listed thinking models extra completion budget and, where the vendor documents it, turns thinking off (Anthropic, DeepSeek V4, Moonshot/Zhipu thinking-type APIs) or uses the lowest documented GPT-5 effort. Custom or unlisted History Compress models get the same extra budget without inventing vendor-specific fields. Background compaction stores that model's configured context window on the job, and emergency overflow recovery resolves the same configured window from your runtime model card (so an 8k custom summarizer is not treated as 128k in either path). ChatHub never copies hidden reasoning into the stored topic summary. Durable message translation is not capped by that summary limit. Long tool/code turns are split on complete user/assistant turns so each summary request stays under about half of that model's context window (DeepSeek V4 Flash is 1M). A complete turn that still exceeds that budget is skipped with a short placeholder in the topic summary so Compact can advance past huge tool/code dumps instead of failing the whole run. If many of those placeholders accumulate, ChatHub keeps the newest part of the running summary. Prefer a capable History Compress model; pointing it at the same overloaded chat model can still make budget-legal batches fail.
• Empty summaries — if the summarizer still returns no visible text, or the provider rejects the prompt as over the context window (maximum context length is …), a background compaction job fails once instead of retrying the same request eight times. Pre-send compact in the browser reports a failed compaction and leaves history unchanged. Every summarizer call also runs under a 120-second deadline (server CONTEXT_COMPACTION_SUMMARIZER_TIMEOUT_MS, browser NEXT_PUBLIC_CONTEXT_COMPACTION_SUMMARIZER_TIMEOUT_MS): a hung summarizer fails that compaction once instead of blocking your send forever. See Background Conversation Generation.
• Automatic overflow retry — if the chat model still rejects a request as too long (for example context_length_exceeded, maximum context length, prompt is too long, MiniMax 2013) or returns an empty reply at the window ceiling, ChatHub does not just show the error: it runs one compaction and resends your message once, in both background (durable) turns and in-tab generation. This emergency recovery runs even when Limit history messages / Summarize history are off — those switches govern routine scheduled and manual compaction, not rescuing a send the provider already rejected. The recovery compaction uses the configured History Compress model (Settings → System Agent), never the chat model, and the retry rebuilds the request from the current conversation, so tool results the failed attempt already produced are kept rather than re-run, knowledge retrieved for your message is kept too, and the resend actually uses the fresh summary even with the routine switches off. You only see the context-length error bubble when compaction cannot reclaim enough history; a send is never retried more than once.
• Tool results — ChatHub sends the full stored tool body to the model, including MCP tools/call results. It does not rewrite long results with a …[truncated N chars] marker. Token auto-compact still runs when the next-request estimate crosses the high watermark. If one result is so large the provider rejects the next request, the automatic overflow retry still compact-and-resends once.
• Empty reply at the window — if the chat model accepts a request that already fills its listed window and returns no text (provider input tokens ≈ the 1M card, output tokens 0), ChatHub marks that bubble as a context-window error instead of a successful blank reply. That applies to both background (durable) turns and in-tab generation when background jobs are off or fall back to the browser. ChatHub then tries compact again (a previous failed compact is not stuck-replayed). Send stays blocked only while usage is still at/above the high watermark after that compact; if Compact cannot reach the low watermark but usage is already below your configured high watermark (80% by default), you can still send.
• Pre-send history overflow — before a server-mode send enqueues a durable job or builds a browser request, ChatHub runs message_count compaction when the history window would drop settled turns, then token_threshold. That compaction is scoped to the conversation being sent, so opening another topic while auto-create-topic is in flight still summarizes the new topic's history rather than the one you just opened. Auto-create-topic sends create (or resolve) the topic first, compact that history, then enqueue so the first topic request is not truncated without a summary. If you switch to another assistant session while topic creation is still in flight, ChatHub still hydrates the created topic in the source session (scoped refresh + local pin) and uses that session's assistant settings for compaction. Topic creation uses a stable client topic id (clientId) scoped to the account, source session/group, and clear fence so a committed create can be retried after a lost response without duplicating Default Topics or reusing another session's identity. If topic creation fails before a topic id exists, ChatHub restores the editor and shows a send error. If creation already committed, the send continues against that topic (including a generated title instead of leaving Default Topic). If the topic cannot be loaded for compaction after create, the send is blocked instead of enqueueing uncovered history. After compaction, ChatHub re-checks account ownership and conversation-clear fences before the server mutation; switching topic or session alone does not cancel the send. Post-send message_count remains as catch-up. Both pre-send paths honor Stop and process at most three summary batches before the send continues.
• Large context windows — when the usable input budget is at least 128k tokens, ChatHub may expand (or temporarily disable) the message-count truncate while the next-request chat payload still fits under about 55% of that budget after fixed overhead (system role, tools, assistant memory, wrapped history summary, activated skill XML) plus per-user input-template expansion on included chat rows and the unsent editor draft or a pending file-only send (the same {{text}} template the request will apply). The input budget is the model context window minus the completion this request will reserve: the max tokens value when that switch is on, otherwise the model's listed maximum output when that maximum is lower than the window. A saved max tokens number is not reserved, and is not sent to the provider, while the switch is off. Background generation follows that switch the same way an in-tab send does. The learned correction updates when a completed reply reports usage, including a background reply and usage stored on the message metadata. Typing or streaming does not reload it. A reply whose skill text was not already available is skipped for that correction, and the send still goes out. A thread counts the earlier topic messages that request will send. The token popover's included of topic line counts persisted topic rows only (not the synthetic draft row). Smaller budgets keep the configured history limit. Compaction still runs when overflow appears.
• Token auto-compact — reacts when the local token estimate or conversation changes; it does not repeatedly poll an unchanged chat. The 80% (configurable) high-watermark gate applies only to this token-threshold path. It compares a client estimate of the sliced request (system role, tools, memory, summary, sliced history, images, input) to that input budget — not the full topic against the raw context window. OpenAI-compatible cards stay on their 258k window, then subtract the listed output reserve. message_count and Compact now ignore that 80% gate and can compact a short chat. At the high watermark it summarizes the oldest completed turns needed to target the low watermark. The estimate includes assistant memory plus enabled archive excerpts so it matches request context more closely. Guarded background compact (token auto-compact after send) can finish after you close the tab. If a compact fails, ChatHub retries with a new background job instead of stuck-replaying the failed one. If Compact reports that the protected turn cannot reach the low watermark (target unreachable), it does not keep retrying the same unchanged chat; a new message, summary, or model change can arm it again. Send is blocked only while usage is still at/above the high watermark after compact (you get the usual context-length error bubble, and Compact can run again). See Background Conversation Generation.
• Why did it compact early? — set CHATHUB_COMPACTION_DEBUG=1 on the container and recreate it (no image rebuild). Then filter Axiom for debug_namespace=chathub-compaction-debug. planner_settled records trigger, status, reason, ratio, maxTokens, window counts (topicMessageCount, effectiveHistoryCount as the history-limit setting, excludedByHistoryCount), preSendMessageCountCompact, and the token parts. Dream scheduler events (dream_scheduler_tick / dream_scheduler_settled, path=assistant_memory_rollup) live in the same switch; dated-card settle records include keep-N counts, overflow envelope kind, and whether overflow used the LLM rewrite or concat fallback. See Debug and Diagnostics.
• Daily memory dream — runs on the server at the configured UTC time (no open browser). It reads yesterday's active topics — the compaction summary when one exists, otherwise a bounded recent-message excerpt from that day. After Daily topic note was removed, planner_settled with trigger=scheduled on a topic path is a regression.
• Scope — topic compaction and topic-summary injection run only in regular topic chats. Group chats, active threads, and portal threads keep their own raw scoped history and do not use a regular topic summary; assistant-wide memory still applies. The portal token popover does not show the manual button.
• Raw history — compaction does not delete messages. A per-topic cursor records which completed turns are represented by the rolling summary, and only newer messages are merged on the next run.
• Protected content — the latest user turn and any unresolved assistant/tool continuation remain verbatim. If that protected context plus fixed instructions is already above the low watermark, the result is reported as target unreachable.
• Long tool continuations — history limiting stays anchored to the latest user turn. In the pathological case where its assistant/tool tail exceeds 20 messages, ChatHub keeps the newest 20 so the active tool loop retains its latest results.
• Stop and send ordering — Stop cancels pre-send compaction as well as generation. A cancelled compaction does not leave its new summary active, and server-mode sends create the assistant placeholder only after the compaction attempt settles without cancellation and the conversation is still current. Stop in a portal thread is scoped to that portal thread and does not cancel main-chat compaction for the same topic.
• Debug panel — after compaction, metadata stores a short log (memoryDebugLog) with the trigger, result, high/low watermarks, estimates, and compacted-through message ID.
• Archives — optional excerpts appended under the history summary block. Identical snapshots are deduplicated. If an old or malformed archive timestamp cannot be formatted, the excerpt is still kept with an unknown time marker.
• Edits, deletes, and retries — changing or removing history already represented by the cursor clears the derived topic summary and snippets. A later run rebuilds them from the retained raw history.

Assistant Memory (per agent, cross-session)

• Each assistant has separate fixed and dynamic memory stored in PostgreSQL. Both are injected before the active topic's compaction summary when the assistant-memory master switch is on.
• Fixed memory changes only through user or model memory actions. Automatic rollups treat it as context and do not rewrite it.
• The scheduled Daily memory dream is a Graphile job. It runs on the server at the configured UTC time and reads topics that were active on the previous UTC calendar day — the compaction summary when one exists, otherwise a bounded excerpt of that topic's recent messages from that day. Each successful run appends a new #N [YYYY-MM-DD]: card to dynamic memory (weekly schedule still tags the previous UTC calendar day). Use per-card Regenerate to re-run the dream for one day; merged range cards and legacy blobs cannot be regenerated.
• Keep dream cards (memoryDreamMaxEntries, default 14) retains the newest N single-day cards. When the dream runs, older single-day cards are re-summarized into one range card (up to 6400 characters). If that summary is over 6400 after headers, or the provider stops at a token limit, ChatHub asks the model to rewrite it shorter rather than keeping a truncated draft. Saving the Memory form applies retention and the new keep value atomically; a settings-only fold concatenates then caps at 6400 until the next dream can re-summarize. Opaque overflow (cards ChatHub cannot split into dated sections) keeps its newest tail when another day is folded, even if that day's text looks like a date heading.
• Dynamic memory keeps one previous-version backup internally. A successful no-change run does not append a card.