@ session autocomplete latency grows linearly with total session count — full corpus scanned (incl. zstd header decompression) on every keystroke, no cache/limit

[lws2004]

Summary

Typing @ in the Web GUI triggers session autocomplete that enumerates the entire persisted session corpus on every keystroke, with no caching or bounds. Latency grows linearly with session count; at ~9.5k sessions it becomes hundreds of ms to seconds per keystroke and competes with the agent loop on Node's single thread.

Environment

• @deepseek-ai/dsh@0.1.1-rc.2 (jsonl session persistence, default)

Failure path

Every @ input calls remote.sessionReferenceResolver.candidates (dsh-client-ui-reference):

1. dsh-session-reference → sessionQuery.listSessions → SessionPersistence.listArtifacts (dsh-session-persistence-jsonl), which for every stored session does:
   • stat existence checks,
   • zstd header decompression (readFirstZstdLine) to parse the header line,
   • identity assertions + id dedup.
2. On non-empty queries, readTitleSnapshots is called for all records, not just the candidateLimit subset.
3. No cache or invalidation (only abort-on-newer-query); no debounce on the client side.

Reproduction

• Accumulate ~9.5k sessions (many of which can be 4-line empty shells — only session/permission/sandbox init events; we observed 8.5k+ created within a 17-minute window, likely by an automated/probe run).
• Each @ keystroke then decompresses ~9.5k zstd files (~10 MB) synchronously inside the host process, serialized with the agent loop.

Key contrast

• File completion is bounded by maxEntries (default 10000) and excludes .git/node_modules.
• Session enumeration has no equivalent cap: candidateLimit (default 50) only trims returned results, never the scan itself.

Suggested directions

• Cache the corpus listing with explicit invalidation (tool results / filesystem changes), or
• Maintain a lightweight header index so listing doesn't require decompressing every session log, or
• Serve @ completion from a bounded projection (e.g. only most-recent N sessions per project) without a full scan.

Workaround

Bulk-archiving old/empty sessions out of ~/.dsh/sessions restores sub-second autocomplete; @" (quoted) file references skip session candidates entirely in the current client code.

[lws2004]

Update after upgrading to @deepseek-ai/dsh@0.1.2-rc.1 (and re-checking current master, post dsh-v0.1.3-alpha.1): the latency is gone on my machine, but the root cause is still present — my earlier relief came from the workaround, not from a fix.

• My local corpus dropped from ~9.5k sessions to 26 (the automated empty-shell runs were cleaned up), so per-keystroke scans are now trivially cheap. This matches the Workaround section above.
• Verified on current master:
  • dsh-session-persistence-jsonl listArtifacts (src/index.ts, ~L731) still walks every project/session directory with no cache and no bound, and still decompresses the first zstd line of every log per session (readGenerationHeader → readFirstZstdLine).
  • dsh-client-ui-reference (client) candidates still issues a fresh full lookup per keystroke — no debounce, no cache (only the quoted @" shortcut skips session candidates).
  • The v0.1.2 release-notes items ("optimize the / and @ menus…", "reduce unnecessary filesystem checks…") improve the UI layer but do not touch the unbounded full-corpus scan.

Keeping this open since the suggested directions (cache with explicit invalidation, a lightweight header index, or a bounded projection) still apply. Happy to help test a patch.