dsh-rs: in-process grep/glob for tool-fs-search — no rg spawn, no stdout cap (measured, fork branch)

[Saidoua]

English | 中文

Sharing an ecosystem project, per CONTRIBUTING: dsh-rs, an in-process backend for the grep and glob tools of @deepseek-ai/dsh-tool-fs-search, and the fork that wires it in behind a fallback.

• Addon: https://github.com/Saidoua/dsh-rs (Rust, napi-rs, dsh-plugin topic; MIT)
• Plugin, no fork required: https://github.com/Saidoua/dsh-rs/tree/main/plugin
• Integration, in the fork's master: https://github.com/Saidoua/deepseek-harness

What it does

Both tools spawn the packaged ripgrep binary per call and parse its --json stdout. dsh-rs runs ripgrep's own library crates (grep-regex, grep-searcher, ignore) inside the harness process, on libuv's thread pool, so the tool call does not hold the event loop for the length of a tree walk. The spawn stays as the fallback: DSH_NATIVE=0, a platform with no prebuilt binary, or an unloadable addon keeps today's behaviour exactly. execute calls it uncapped, so the retention and spill layers keep owning the inline caps and previews.

Installing it without a fork

The addon is a compiled napi-rs binary with no Cordis surface, so it cannot be mounted; and a bundle patch composes plugin entries, so it cannot change how an already-registered plugin executes a search. That is why the integration above lives in a fork.

plugin/ in the addon repository closes that gap: @saidouahdachi/dsh-tool-fs-search-native registers grep and glob against the addon and ships a dsh.bundle.patch layer that disables the in-box tool-fs-search entry, so a profile picks the backend up through dsh plugin add. Tool names are exclusive within a scope, which is why the in-box entry is disabled rather than reconfigured. Names, schemas, descriptions, system-prompt guidance, render text, search cards, and spill recovery are all @deepseek-ai/dsh-tool-fs-search's own exports, so a session sees nothing new; only execute differs, and with no usable addon (DSH_NATIVE=0, or a platform with no prebuilt binary) the plugin registers the in-box spawn-backed tools itself.

Checked in real dsh sessions through one profile: the broad search that fails the 20 MB cap on the spawn path answers in-process, and the same profile under DSH_NATIVE=0 fails exactly as an unpatched one does.

The addon is on npm as a family — @saidouahdachi/dsh-native carries the loader and one prebuilt package per target (darwin arm64/x64, linux x64/arm64 glibc, win32 x64) rides as an os/cpu-gated optional dependency, so an install pulls the matching binary or none at all and an uncovered host simply spawns ripgrep. The plugin is on npm too, so a profile picks the whole thing up with dsh plugin --profile <name> add @saidouahdachi/dsh-tool-fs-search-native and the patch layer it ships; the harness packages it compiles against stay optional peers, supplied by the installation it mounts into.

Why: a capability, not a percentage

This is the same failure #4407 reports. grep "const" over packages/ — an ordinary agent search — makes ripgrep emit 21.1 MB of --json, over the tool's 20 MB rawOutputMaxBytes, so the spawn path fails with SEARCH_RAW_OUTPUT_OVERFLOW and the model is told to narrow the search. In-process there is no transport to overflow:

grep "const" in packages/ — 21 MB of rg --json, over the tool's 20 MB cap:
  spawned rg : FAILS  SEARCH_RAW_OUTPUT_OVERFLOW  (68 ms)
  dsh-rs     : ok     Found 250 of 75108 matches  (818 ms)

#4407's spill-backed parse is the right in-tree fix for the transport; this removes the transport. It also sidesteps the spawn-environment failures in #3867 (no subprocess to spawn) and #5260 (no rg binary to locate).

Latency is the smaller half of the case, measured through ctx.tools.execute on both backends, median of 7 (Apple M2 Pro, Node 25.9):

Tool call	spawned rg	dsh-rs
grep, packages/session	11.0 ms	6.2 ms (1.77x)
glob *.ts, packages/session	6.1 ms	2.8 ms (2.20x)
grep / glob, whole packages/ tree	58 / 69 ms	52 / 65 ms (~1.1x)

That is 2–5 ms per call inside turns that spend seconds in the model. Ship it for the failure mode, not the milliseconds.

Parity

ripgrep matches every glob against the path it prints (the path argument joined with the walked suffix); the addon reconstructs that form and routes globs through the walker's own ignore::overrides rooted at the workdir, so ripgrep's precedence (an explicit positive --glob outranks the hidden-file filter) holds. A 13-check suite compares against the packaged @vscode/ripgrep binary on identical argv: match sets over a fixture tree and over this repo's source tree, every target shape, separator-anchored include globs, per-file contiguity, VCS-directory targets, literal-newline rejection, and the --sort=modified mtime sequence. The package's real-filesystem integration suite runs once per backend; the full suite is green at 0.1.3-alpha.1.

One contract difference, documented: the cooperative tool timeout terminates the rg process tree, and a running in-process search has no process to terminate, so exec.signal is honoured before the search starts and observed when it returns.

Also on the fork

A small session-resilience plugin set on documented seams, no agent-loop changes, each priced against the request surface it adds: session-rules (/rule + rule_pin, a digest-checked <pinned_rules> message republished verbatim after compaction; 140 tokens of tool schema per request), tool-state (state_write, one replacing <task_state> snapshot; 243 tokens), skill-stats (log-only usage projection), skill trust marking (untrusted-source skills render behind a caution; trust is asserted, never assumed), seed-turn sparing in compaction-basic, and an unmounted compaction-prefix-slide (eviction without a summarizer call — measured 5.3 s and ~13 000 prompt tokens per compaction saved). The Agent Note in the fork records the measurements and the alternatives rejected.

Happy to answer questions here. Not a PR, per the contributing guide.

---

中文

按 CONTRIBUTING 分享一个生态项目：dsh-rs，@deepseek-ai/dsh-tool-fs-search 中 grep 与 glob 两个工具的进程内后端，以及把它接进去、保留 fallback 的 fork。

• Addon：https://github.com/Saidoua/dsh-rs （Rust、napi-rs、dsh-plugin topic；MIT）
• 插件，不需要 fork：https://github.com/Saidoua/dsh-rs/tree/main/plugin
• 集成，已经在 fork 的 master 上：https://github.com/Saidoua/deepseek-harness

它做什么

两个工具每次调用都会 spawn 打包的 ripgrep 二进制，并解析它的 --json stdout。dsh-rs 把 ripgrep 自己的库 crate（grep-regex、grep-searcher、ignore）跑在 harness 进程内、libuv 的线程池上，因此一次工具调用不会在整棵目录树遍历期间占住事件循环。spawn 保留为 fallback：DSH_NATIVE=0、没有预编译二进制的平台、或加载不了的 addon，行为与今天完全一致。execute 以不设上限的方式调用它，内联上限与预览仍由 retention 与 spill 两层负责。

不用 fork 也能装上

addon 是编译好的 napi-rs 二进制，没有 Cordis 侧的接入面，因此无法被挂载；而 bundle patch 只负责组合插件条目，改不了一个已注册插件执行搜索的方式。上面的集成之所以放在一个 fork 里，原因就在这里。

addon 仓库里的 plugin/ 补上了这一段：@saidouahdachi/dsh-tool-fs-search-native 把 grep 与 glob 注册到 addon 上，并附带一层 dsh.bundle.patch，把内置的 tool-fs-search 条目禁用掉，于是一个 profile 通过 dsh plugin add 就能用上这个后端。工具名在一个作用域内是唯一的，所以内置条目是被禁用，而不是被重新配置。名称、schema、描述、系统提示词引导、渲染文本、搜索卡片与 spill 恢复，全部来自 @deepseek-ai/dsh-tool-fs-search 自己的导出，因此会话侧看不出任何变化；只有 execute 不同，而在没有可用 addon 时（DSH_NATIVE=0，或没有预编译二进制的平台），插件会自行注册内置的 spawn 版工具。

已在同一个 profile 上用真实 dsh 会话验证：在 spawn 路径上撑爆 20 MB 上限的那次宽搜索，进程内可以正常返回；而同一个 profile 加上 DSH_NATIVE=0 之后，失败方式与未打补丁的 profile 完全一致。

addon 已经以一个包家族发布到 npm：@saidouahdachi/dsh-native 承载 loader，每个目标平台一个预编译包（darwin arm64/x64、linux x64/arm64 glibc、win32 x64）作为受 os/cpu 收窄的可选依赖，因此安装时要么装上匹配的二进制，要么一个都不装；不在覆盖范围内的宿主就直接 spawn ripgrep。插件也已经发布到 npm，因此一个 profile 用 dsh plugin --profile <name> add @saidouahdachi/dsh-tool-fs-search-native 加上它自带的 patch 层就能装好；它编译时依赖的 harness 包保持为可选 peer，由它挂载进去的那个安装提供。

为什么：这是能力差异，不是百分比

这正是 #4407 报告的那个失败。grep "const" 扫 packages/ —— 一次再普通不过的 agent 搜索 —— 会让 ripgrep 输出 21.1 MB 的 --json，超过工具 20 MB 的 rawOutputMaxBytes，于是 spawn 路径以 SEARCH_RAW_OUTPUT_OVERFLOW 失败，模型被告知缩小搜索范围。进程内没有可被撑爆的传输通道：

grep "const" in packages/ — 21 MB of rg --json, over the tool's 20 MB cap:
  spawned rg : FAILS  SEARCH_RAW_OUTPUT_OVERFLOW  (68 ms)
  dsh-rs     : ok     Found 250 of 75108 matches  (818 ms)

#4407 的 spill 落盘解析是针对这条传输通道的正确树内修复；这里则是把传输通道整个去掉。它同时绕开了 #3867 与 #5260 中的 spawn 环境失败（没有子进程要 spawn，也没有 rg 二进制要定位）。

延迟是这个论证里较小的一半，在两个后端上都通过 ctx.tools.execute 实测，7 次取中位数（Apple M2 Pro，Node 25.9）：

工具调用	spawn rg	dsh-rs
grep，packages/session	11.0 ms	6.2 ms（1.77x）
glob *.ts，packages/session	6.1 ms	2.8 ms（2.20x）
grep / glob，整棵 packages/ 树	58 / 69 ms	52 / 65 ms（约 1.1x）

也就是在动辄花掉数秒模型时间的轮次里，每次调用省下 2–5 ms。为那个失败模式而 ship 它，不是为这几毫秒。

一致性

ripgrep 用它打印出来的那个路径去匹配每个 glob（path 参数与遍历得到的后缀拼接而成）；addon 重建这一形式，并把 glob 交给 walker 自己的、以 workdir 为根的 ignore::overrides 处理，因此 ripgrep 的优先级规则（显式的正向 --glob 压过隐藏文件过滤）依然成立。一套 13 项检查在相同 argv 下与打包的 @vscode/ripgrep 二进制对照：fixture 目录树与本仓库源码树上的匹配集合、每种 target 形态、以分隔符锚定的 include glob、单个文件内的连续性、VCS 目录 target、字面换行的拒绝，以及 --sort=modified 的 mtime 顺序。包自带的真实文件系统集成用例在每个后端各跑一遍；0.1.3-alpha.1 上全套是绿的。

有一处约定差异，已写进文档：协作式工具超时会终止 rg 的进程树，而进行中的进程内搜索没有进程可以终止，因此 exec.signal 在搜索开始前被遵守，在搜索返回时被观察到。

fork 上还有什么

一组小的会话韧性插件，都建立在有文档的扩展点上，不改动 agent-loop，每个都按它增加的请求开销计价：session-rules（/rule + rule_pin，一条带摘要校验的 <pinned_rules> 消息在压缩后原样重新发布；每次请求 140 个 token 的工具 schema）、tool-state（state_write，一份会被整体替换的 <task_state> 快照；243 个 token）、skill-stats（只写日志的使用情况投影）、skill 信任标记（来源不受信的 skill 带警示渲染；信任是被声明的，从不假定）、compaction-basic 中对种子轮次的保留，以及一个未挂载的 compaction-prefix-slide（不调用摘要器的淘汰 —— 实测每次压缩省下 5.3 秒与约 13 000 个 prompt token）。fork 里的 Agent Note 记录了这些测量与被否决的替代方案。

欢迎在这里提问。按贡献指南，这不是一个 PR。