Performance: zero prompt-cache hits + large tool-schema overhead slow down long tasks

[zeros335882878]

Observation

Long-running tasks (e.g. asking dsh to read and explain a paper from a GitHub repo) get progressively slower. In one session, total tokens grew from 25K to 56K across turns, and wall-clock per turn climbed from ~3.5s to ~53s.

Data (Tencent copilot backend, deepseek-v4-pro, dsh 0.1.0-rc.6)

Every turn's usage shows prompt cache hits are always 0:

"prompt_tokens_details": { "cached_tokens": 0 },
"prompt_cache_hit_tokens": 0

Each turn re-processes ~11-12K prompt tokens from scratch. The request always carries the full standard-mode toolset (~30 tools, roughly 6.6K tokens of JSON schema) plus the growing message history, so the prompt prefix is not stable enough to hit the backend's prompt cache.

Impact

• No cache acceleration -> every turn pays the full prompt cost and latency
• Large fixed tool-schema overhead on every turn
• Context grows unboundedly on multi-step tasks, so later turns are the slowest

Suggestions

1. Lazy / selective tool loading: send only the tools relevant to the current mode or turn instead of the full ~30-tool schema every turn
2. Keep the prompt prefix stable (system prompt + tool schema ordering) so backend prompt caching can hit across turns
3. Context compaction for long-running sessions

Would be great to know if the team has plans around tool-schema trimming or prefix-stable caching. Thanks!

[Suertida]

补充一个更普遍、和长任务无关的视角：schema 是每轮固定成本，而且和工具的"散文段"重复。

我在 DSH Desktop 2.0.2（desktop profile）实测：系统提示词里除了工具 schema，还有一堆工具散文段（tool-guidance prose）会注入系统提示词。一共统计了14 处、来自 9 个官方包（order 99–190），累计占比超过 persona。而且散文段大量复述 schema description 里已有的内容（例：web 工具在 dsh-tool-web 和 dsh-web-access 各有一套说明，同一信息出现两遍）。

我把 14 处散文全删掉之后，剩下工具 JSON schema 的占比仍然很高，楼主说的 trimming 是下一步的真实需求。

建议允许用户自定义编辑每个工具的description。

---

To add a broader perspective that is not specific to long-context tasks: the schema represents a fixed per-round cost, and it also overlaps with the tool "prose paragraphs."

In my tests on DSH Desktop 2.0.2 (desktop profile), besides the tool schemas, a bunch of tool-guidance prose is also injected into the system prompt. In total, I counted 14 prose entries from 9 official packages (order 99–190), with a cumulative size exceeding that of the persona section. Moreover, much of this prose merely restates content already present in the schema descriptions (for example, the web tool has two separate sets of instructions—one in dsh-tool-web and another in dsh-web-access—so the same information appears twice).

After I removed all 14 prose entries, the remaining JSON schemas for the tools still accounted for a significant portion. So the trimming mentioned by the OP is indeed a real need for the next step.

I would suggest allowing users to customize the description field for each tool.