🔎 dsh-llm-trace-plugin — see the literal bytes on the wire between Harness and the LLM API

[lastorder]

English version

🔎 dsh-llm-trace-plugin — see the literal bytes on the wire between Harness and the LLM API

What it does: patches globalThis.fetch to capture the exact HTTP request/response of every LLM provider call Harness makes — raw headers, raw JSON bodies, raw SSE frames — and shows them in a Wire Trace tab right inside the session UI. No proxy, no mitmproxy, no extra process.

Why it's useful for learning Harness ↔ LLM API:

• See precisely what dsh-llm-deepseek / dsh-llm-pi-ai actually send: real provider field names, real headers, the real streaming frame sequence — not the normalized GenerateOptions/StreamChunk objects Harness builds on top.
• Every record is tagged with Turn / Step / Session / provider / purpose (session-title vs compaction vs normal turn), attached via llm/stream + session/event, so you can map "this exact wire call" back to "this exact point in the conversation loop" — without reading Harness internals.
• A jq-shaped, foldable JSON viewer (global depth stepper + per-row fold) makes even huge 1M-token request bodies and reassembled SSE deltas actually readable.

Why it's useful for debugging LLM API issues:

• A curl-replayable capture of the literal request — reproduce a failing call outside Harness entirely, headers and body included (credentials redacted, obviously).
• SSE responses are reassembled from scattered delta fragments back into one complete structure, so you don't have to manually stitch chunks to see what the model actually said.
• Records persist to disk (one JSON file per record, atomic write+rename, no shared-file corruption risk) so a crash doesn't lose the trace you most need.
• Transport failures and HTTP errors are recorded and rethrown unchanged — nothing is swallowed, so the trace matches what Harness itself experienced.
• Filter by current session or see everything, including subagent sessions.

Install:

dsh plugin --profile web add dsh-llm-trace-plugin

Restart dsh web after installing.

Repo / docs: https://github.com/lastorder/dsh-llm-trace-plugin (bilingual README, architecture doc, plugin-development notes)

Category: Debugging / Developer Tools

---

中文版本

🔎 dsh-llm-trace-plugin —— 直接看到 Harness 与 LLM API 之间线上的原始字节

它是什么： 通过替换 globalThis.fetch，捕获 Harness 发出的每一次 LLM provider 调用的真实 HTTP 请求/响应——原始 header、原始 JSON body、原始 SSE 帧——并在 session 界面内的 Wire Trace 标签页里直接查看。不需要代理、不需要 mitmproxy、不需要额外进程。

为什么适合用来学习 Harness 与 LLM API 的关系：

• 精确看到 dsh-llm-deepseek / dsh-llm-pi-ai 实际发送了什么：真实的 provider 字段名、真实的 header、真实的流式帧序列——而不是 Harness 在此之上构建的、provider 无关的 GenerateOptions/StreamChunk 对象。
• 每条记录都标注了 Turn / Step / Session / provider / purpose（session-title、compaction、还是普通对话轮次），这些信息通过 llm/stream 与 session/event 两个公开扩展点观察得到——不需要阅读 Harness 源码，就能把"这次具体的线上调用"对应回"对话循环中的具体位置"。
• jq 风格、可折叠的 JSON 查看器（全局深度步进 + 逐行折叠），让哪怕是百万 token 级的超大请求体、以及重新拼装后的 SSE 增量，也能真正可读。

为什么适合用来 debug LLM API 相关问题：

• 捕获的原始请求可直接 curl 重放——完全脱离 Harness 复现失败调用，header 和 body 都在（凭证 header 已脱敏）。
• SSE 响应会把分散的 delta 片段重新拼装成一份完整结构，不用再手动拼接 chunk 才能看清模型到底说了什么。
• 记录会持久化到磁盘（每条记录一个 JSON 文件，原子写入+改名，没有共享文件损坏的风险），进程崩溃也不会丢失你最需要的那条 trace。
• 传输失败和 HTTP 错误会被完整记录并原样重新抛出——不会被吞掉，trace 与 Harness 实际经历的完全一致。
• 可按当前 session 过滤，也可以查看全部记录，包括子代理（subagent）的会话。

安装：

dsh plugin --profile web add dsh-llm-trace-plugin

安装后需要重启 dsh web。

仓库 / 文档： https://github.com/lastorder/dsh-llm-trace-plugin （中英双语 README，架构文档，插件开发说明）

分类：调试 / 开发者工具

[argszero]

Nice tool — real wire-level visibility without a proxy is a genuinely useful gap to fill, and riding llm/stream + session/event to tag records is exactly the right seam use. Two engineering notes from the harness side that may save you debugging time:

1. Capture coverage is only as good as the globalThis.fetch patch. The harness's own DeepSeek adapter calls a bare global fetch (packages/llm/llm-deepseek/src/adapter.ts:643), so it will appear in your trace — but the ecosystem has a documented history of fetch/undici global-slot interference: Node's built-in fetch and npm undici share Symbol.for('undici.globalDispatcher.*'), and merely import('undici') anywhere in the process can swap the dispatcher under you (family of reports: web_fetch overriding the global dispatcher, setGlobalDispatcher no-clobber being defeated by import side-effects, LLM ~302s body-timeout deaths from dispatcher/header config). Consequences for a fetch-patching plugin:

• Load order matters: if anything re-assigns globalThis.fetch after your patch (a later import of undici, another plugin), you silently stop seeing calls — no error.
• Non-global egress escapes: any provider/client that imports undici's fetch directly, pins a per-request dispatcher, or runs fetch in a worker/isolate bypasses the patch entirely. dsh-llm-pi-ai routes through the pi-ai SDK, whose transport you should verify actually lands on the global.
  A cheap "coverage health" signal (e.g. count calls per provider and warn when a provider that should be active produced zero captures) would make the trace trustworthy rather than assumed-complete.

2. Redaction scope: "credentials redacted" — make sure that covers URL query strings and authorization/cookie headers (the harness mints a per-process launch token that can ride URLs, and browser-session cookies carry a signed authority). A jq-style viewer that shows a raw cookie header can leak the session secret into screenshots/saved records.

Also worth stating in the README: because this patches the process-global fetch, it belongs in a debugging profile and should probably refuse to load alongside other fetch-mutating plugins — the single-global-slot constraint is real.

Solid contribution to the debugging niche — the ecosystem benefits from more "see what actually happens" tools like this (cf. dsh-llm-oauth from #5845, subagent-wait, etc.).

[lastorder]

Thanks for the feedback! I’ve made some updates based on your suggestions and released a new version with these changes. Really appreciate your input!