fix: bound provider telemetry memory

docs-web/architecture/high-concurrency-orchestration.md

@@ -101,6 +101,12 @@ the dashboard, and interactive replies.
 - Incremental Codex telemetry follows the native thread id emitted by the current exec stream (or
   the exact requested resume id) and reads only that rollout from the paired runtime volume. A
   previous invocation's newer file cannot replace the persisted continuation identity.
+- Codex rollout parsing skips any single JSONL record above 2 MiB, truncates retained transcript
+  fields, and keeps the newest 256 conversation item groups. Oversized generated assets or command
+  output cannot exhaust the server heap, and parsing resumes at the next record.
+- Jules full-history telemetry is serialized process-wide, joins duplicate in-flight session reads,
+  tokenizes text in slices of at most 64 KiB, and releases raw activities before SQLite
+  reconciliation. Wide hosted-session syncs therefore cannot multiply large patch payloads in heap.
 - Antigravity sends `--conversation` only for provider-native ids. Logical/workspace continuation
   markers use `--continue` inside the isolated paired runtime volume instead.
 

docs-web/content/docs/architecture-high-concurrency-orchestration.mdx

@@ -101,6 +101,12 @@ the dashboard, and interactive replies.
 - Incremental Codex telemetry follows the native thread id emitted by the current exec stream (or
   the exact requested resume id) and reads only that rollout from the paired runtime volume. A
   previous invocation's newer file cannot replace the persisted continuation identity.
+- Codex rollout parsing skips any single JSONL record above 2 MiB, truncates retained transcript
+  fields, and keeps the newest 256 conversation item groups. Oversized generated assets or command
+  output cannot exhaust the server heap, and parsing resumes at the next record.
+- Jules full-history telemetry is serialized process-wide, joins duplicate in-flight session reads,
+  tokenizes text in slices of at most 64 KiB, and releases raw activities before SQLite
+  reconciliation. Wide hosted-session syncs therefore cannot multiply large patch payloads in heap.
 - Antigravity sends `--conversation` only for provider-native ids. Logical/workspace continuation
   markers use `--continue` inside the isolated paired runtime volume instead.
 

docs/architecture/high-concurrency-orchestration.md

@@ -212,6 +212,10 @@ Codex rollout transport binds every read to the native thread id emitted by the
 `codex exec --json` stream (or the exact requested resume id); it never treats an unrelated newest
 rollout in a reused runtime home as the invocation's identity. Parsing retains a byte cursor,
 handles split UTF-8/JSONL records, caps work per poll, and resets on source rotation or truncation.
+The parser discards any single JSONL record above 2 MiB, bounds retained message/tool fields, and
+keeps only the newest 256 conversation item groups. Large generated assets or command output can
+therefore remain in the provider-owned rollout without exhausting the server heap; normalized usage,
+session identity, and later records continue to be processed.
 Claude transport also reads appended bytes.
 Qwen mutable JSON files and the Antigravity SQLite source use a coherent full read only after their
 cheap metadata changes; unchanged polls do not copy or parse them.
@@ -234,7 +238,12 @@ Structured invocation messages reconcile by stable ordinal in one SQLite transac
 
 Text-only completion fallback remains append-only so retry and audit messages are not removed.
 Hosted Jules activity-to-message sync uses the same atomic suffix reconciliation instead of clearing
-and reinserting the invocation transcript on each poll.
+and reinserting the invocation transcript on each poll. Full Jules histories are fetched and
+tokenized through a one-at-a-time process queue, duplicate in-flight requests for the same session
+join the existing work, and tokenizer input is sliced to at most 64 KiB. The raw activity array is
+released after bounded messages and numeric usage are derived, before SQLite reconciliation begins.
+This keeps wide hosted-session synchronization from multiplying large patch and media payloads in
+the Node.js heap.
 
 Streaming provider activity is buffered for 250 ms or 50 source records, then adjacent records from
 the same originator are compacted into bounded 16 KiB rows before one batch transaction. This avoids

docs/architecture/usage-telemetry-and-stats.md

@@ -126,6 +126,12 @@ initial target. Code UX derives the rollout date from that id and reads only
 lookup remains only as a compatibility fallback when a legacy caller has no native id, so prior
 sessions in the same runtime home cannot overwrite the persisted continuation identity.
 
+Live rollout parsing is bounded independently of rollout file size. A JSONL record larger than
+2 MiB is skipped until its terminating newline, per-turn text/tool payloads are truncated with an
+explicit marker, and only the newest 256 conversation item groups remain in memory. This protects
+the runtime from generated binary assets and extreme command output while preserving cumulative
+usage snapshots, native session identity, and every later well-formed record.
+
 ### Qwen Code
 
 Qwen Code runs via its OpenAI-compatible request/response logging (`enableOpenAILoggingDir`), written to a directory that is reset at the start of every run so usage aggregation only ever sums the current invocation's own log files — unlike Codex/OpenCode, there is no cross-run cumulative counter to isolate.
@@ -221,7 +227,18 @@ Stats pricing still prefers configured model-pricing overrides and catalogue tok
 
 ### Jules
 
-Jules does not expose a compatible native token contract. Instead of excluding it, Code UX computes **estimated** tokens for Jules by accumulating input and output characters divided by 4 (the characters-per-token heuristic).
+Jules does not expose a compatible native token contract. Instead of excluding it, Code UX computes
+**estimated** tokens from the cumulative activity stream. The estimator models replayed input
+context, generated messages, progress/tool turns, and added lines from patch artifacts. It uses the
+`cl100k_base` tokenizer in slices of at most 64 KiB so a multi-megabyte patch cannot create one
+unbounded tokenizer allocation.
+
+Full-conversation usage synchronization is process-wide serialized. Calls for the same session join
+the existing in-flight sync, while different sessions wait on a one-at-a-time queue. Once bounded
+invocation messages and numeric usage have been derived, Code UX releases the raw remote activity
+array before reconciling SQLite rows. This prevents a wide set of hosted sessions from retaining and
+tokenizing multiple large histories concurrently without changing the persisted message or usage
+contract.
 
 During live synchronization (`syncLiveInvocation`), expected 404 responses indicating that a session or activity stream is unavailable are handled gracefully: they are logged at the debug level and skipped to avoid spamming the logs with warnings. For terminal sync (`calculateAndSaveUsageForTask`), the system is conservative: if the session returns a 404, it skips creating a new usage record to prevent saving "fake" empty records unless an existing prompt or usage record is already present to allow safe estimation.
 

src/domain/jules/jules-usage-estimator.ts

@@ -38,6 +38,11 @@ export const JULES_SYSTEM_PROMPT_TOKENS = 800;
  *  compaction Jules performs and prevents quadratic blow-up on long sessions. */
 export const JULES_CONTEXT_TOKEN_CAP = 200_000;
 
+/** Maximum string slice passed to the tokenizer at once. `js-tiktoken`
+ * materializes token arrays and regex matches, so feeding it a multi-megabyte
+ * patch in one call can temporarily consume gigabytes of V8 heap. */
+export const JULES_TOKENIZER_CHUNK_CHARS = 64 * 1024;
+
 /** Fallback tokens-per-added-line when a diff isn't available but PR git stats are. */
 export const JULES_TOKENS_PER_ADDED_LINE = 12;
 
@@ -80,6 +85,72 @@ export function extractAddedDiffLines(unidiffPatch: string): string {
   return added.join("\n");
 }
 
+/** Tokenizes large values in bounded slices. Jules usage is already an
+ * estimate, and the tiny BPE-boundary variance is preferable to an unbounded
+ * temporary allocation for generated patches and transcripts. */
+export function countJulesTokensInChunks(
+  text: string,
+  countTokens: (chunk: string) => number,
+): number {
+  let total = 0;
+  let offset = 0;
+  while (offset < text.length) {
+    let end = Math.min(text.length, offset + JULES_TOKENIZER_CHUNK_CHARS);
+    if (
+      end < text.length
+      && end > offset
+      && text.charCodeAt(end - 1) >= 0xD800
+      && text.charCodeAt(end - 1) <= 0xDBFF
+    ) {
+      end -= 1;
+    }
+    total += countTokens(text.slice(offset, end));
+    offset = end;
+  }
+  return total;
+}
+
+function measureAddedDiffLines(
+  unidiffPatch: string,
+  countTokens: (text: string) => number,
+): { chars: number; tokens: number } {
+  let batch = "";
+  let chars = 0;
+  let tokens = 0;
+  let hasAddedLine = false;
+  let cursor = 0;
+
+  const flush = () => {
+    if (!batch) {
+      return;
+    }
+    tokens += countJulesTokensInChunks(batch, countTokens);
+    batch = "";
+  };
+
+  while (cursor <= unidiffPatch.length) {
+    const newline = unidiffPatch.indexOf("\n", cursor);
+    const end = newline >= 0 ? newline : unidiffPatch.length;
+    const line = unidiffPatch.slice(cursor, end);
+    if (line.startsWith("+") && !line.startsWith("+++")) {
+      const addedLine = line.slice(1);
+      const separator = hasAddedLine ? "\n" : "";
+      chars += separator.length + addedLine.length;
+      if (batch.length + separator.length + addedLine.length > JULES_TOKENIZER_CHUNK_CHARS) {
+        flush();
+      }
+      batch += separator + addedLine;
+      hasAddedLine = true;
+    }
+    if (newline < 0) {
+      break;
+    }
+    cursor = newline + 1;
+  }
+  flush();
+  return { chars, tokens };
+}
+
 function planToMarkdown(activity: JulesActivity): string {
   const steps = activity.planGenerated?.plan?.steps;
   if (!Array.isArray(steps) || steps.length === 0) {
@@ -102,6 +173,7 @@ function sortByCreateTime(activities: JulesActivity[]): JulesActivity[] {
  */
 export function estimateJulesUsage(input: JulesUsageEstimateInput): JulesUsageEstimate {
   const { countTokens, gitMetrics } = input;
+  const countBoundedTokens = (text: string) => countJulesTokensInChunks(text, countTokens);
   const activities = sortByCreateTime(input.activities || []);
 
   let inputTokens = 0;
@@ -116,7 +188,7 @@ export function estimateJulesUsage(input: JulesUsageEstimateInput): JulesUsageEs
   const prompt = input.prompt || "";
   if (prompt) {
     promptChars += prompt.length;
-    context = Math.min(JULES_CONTEXT_TOKEN_CAP, context + countTokens(prompt));
+    context = Math.min(JULES_CONTEXT_TOKEN_CAP, context + countBoundedTokens(prompt));
   }
 
   const addContext = (tokens: number) => {
@@ -137,24 +209,24 @@ export function estimateJulesUsage(input: JulesUsageEstimateInput): JulesUsageEs
     if (activity.userMessaged?.userMessage) {
       const text = activity.userMessaged.userMessage;
       promptChars += text.length;
-      addContext(countTokens(text));
+      addContext(countBoundedTokens(text));
     }
     if (activity.planApproved?.planId) {
       const text = `Approved plan (ID: ${activity.planApproved.planId})`;
       promptChars += text.length;
-      addContext(countTokens(text));
+      addContext(countBoundedTokens(text));
     }
 
     // Agent-side model output turns.
     if (activity.agentMessaged?.agentMessage) {
       const text = activity.agentMessaged.agentMessage;
       transcriptChars += text.length;
-      billAgentTurn(countTokens(text));
+      billAgentTurn(countBoundedTokens(text));
     }
     if (activity.planGenerated?.plan?.steps) {
       const text = `Proposed plan:\n\n${planToMarkdown(activity)}`;
       transcriptChars += text.length;
-      billAgentTurn(countTokens(text));
+      billAgentTurn(countBoundedTokens(text));
     }
     if (activity.progressUpdated?.title || activity.progressUpdated?.description) {
       // Progress updates are short status lines the agent emits while driving
@@ -164,13 +236,13 @@ export function estimateJulesUsage(input: JulesUsageEstimateInput): JulesUsageEs
       const text = `${title}\n${desc}`;
       transcriptChars += text.length;
       toolCallCount += 1;
-      billAgentTurn(countTokens(text));
+      billAgentTurn(countBoundedTokens(text));
     }
     if (activity.sessionCompleted !== undefined && activity.sessionCompleted !== null) {
-      billAgentTurn(countTokens("Jules session completed successfully."));
+      billAgentTurn(countBoundedTokens("Jules session completed successfully."));
     }
     if (activity.sessionFailed?.reason) {
-      billAgentTurn(countTokens(`Jules session failed: ${activity.sessionFailed.reason}`));
+      billAgentTurn(countBoundedTokens(`Jules session failed: ${activity.sessionFailed.reason}`));
     }
 
     // Code artifacts: the model produced a patch (a tool result). Count only the
@@ -180,15 +252,14 @@ export function estimateJulesUsage(input: JulesUsageEstimateInput): JulesUsageEs
       if (unidiffPatch) {
         sawUnidiffPatch = true;
         toolCallCount += 1;
-        const addedCode = extractAddedDiffLines(unidiffPatch);
-        const codeTokens = countTokens(addedCode);
-        outputTokens += codeTokens;
-        transcriptChars += addedCode.length;
-        addContext(countTokens(unidiffPatch));
+        const addedCode = measureAddedDiffLines(unidiffPatch, countTokens);
+        outputTokens += addedCode.tokens;
+        transcriptChars += addedCode.chars;
+        addContext(countBoundedTokens(unidiffPatch));
       }
       const commitMessage = art.changeSet?.gitPatch?.suggestedCommitMessage;
       if (commitMessage) {
-        const msgTokens = countTokens(commitMessage);
+        const msgTokens = countBoundedTokens(commitMessage);
         outputTokens += msgTokens;
         transcriptChars += commitMessage.length;
         addContext(msgTokens);