Bug: profile package.json with UTF-8 BOM crashes dsh web (Unexpected token ... is not valid JSON)

[LeonisK]

问题现象

在 Windows 上通过源码仓库运行 pnpm dsh web（或 npx @deepseek-ai/dsh web）时，启动立即失败：

SyntaxError: Unexpected token ...
is not valid JSON
at JSON.parse (<anonymous>)
at readProfileManifest (packages/boot/app-boot/src/profile.ts:272:23)
at loadProfile (packages/boot/app-boot/src/profile.ts:385:55)
...
Node.js v24.19.0

复现步骤

1. 安装 DeepSeek Harness（源码 pnpm install / pnpm build，或 npx 安装）；
2. 编辑 profile 的 ~/.dsh/profiles/<profile>/package.json（例如安装插件后手工调整依赖），并用带 BOM 的 UTF-8 保存（部分 Windows 编辑器默认会写入 BOM）；
3. 启动 dsh web，立刻报 is not valid JSON。

根因

packages/boot/app-boot/src/profile.ts 的 readProfileManifest 直接对文件内容调用 JSON.parse，没有先去除文件开头的 UTF-8 BOM（\uFEFF）。JSON 规范不允许在 { 前出现该字符，因此只要 manifest 文件带 BOM 就必然解析失败。

这不是插件或用户配置内容的问题：文件内容本身是合法 JSON，多出来的只是开头 3 个字节（EF BB BF）。

临时解决方法

把 ~/.dsh/profiles/<profile>/package.json 重新保存为 UTF-8（无 BOM）：

• VS Code：右下角编码 → “通过编码保存” → 选择 UTF-8；
• 或命令行去掉 BOM：

node -e "const fs=require('fs');const p=process.argv[1];const b=fs.readFileSync(p);if(b[0]===0xEF&&b[1]===0xBB&&b[2]===0xBF)fs.writeFileSync(p,b.subarray(3))" "$HOME/.dsh/profiles/web/package.json"

建议修复

在 readProfileManifest 读取文件后、JSON.parse 前去掉 BOM：

const text = await fs.readFile(path, 'utf8');
const json = text.charCodeAt(0) === 0xFEFF ? text.slice(1) : text;
return JSON.parse(json);

或更简单：

return JSON.parse(text.replace(/^\uFEFF/, ''));

[zoahdev]

Reproduced and root-caused (Windows, Node 24). The crash is a UTF-8 BOM reaching JSON.parse:

Repro

const raw = Buffer.from([0xEF, 0xBB, 0xBF]).toString() + '{"dependencies":{}}'
JSON.parse(raw) // SyntaxError: Unexpected token '?', "?{"dependencies":{}}" is not valid JSON
JSON.parse(raw.replace(/^\uFEFF/, '')) // works

Root cause
packages/boot/app-boot/src/profile.ts ? readProfileManifest:

raw = readFileSync(path, 'utf8')      // line 267: BOM is preserved by readFileSync
const parsed = JSON.parse(raw) ...    // line 272: JSON.parse rejects U+FEFF

A BOM written by PowerShell (> redirect), Notepad "UTF-8 with BOM", or an editor save option crashes dsh web at boot with Unexpected token ? the user-visible error gives no hint that the file has a BOM.

Minimal fix (one line)

- raw = readFileSync(path, 'utf8')
+ raw = readFileSync(path, 'utf8').replace(/^\uFEFF/, '')

The same class exists at the other JSON.parse(readFileSync(..., 'utf8')) sites in this file (lines 227, 247, 390) ? a shared readJsonFile helper that strips a leading BOM would cover all of them.

Happy to prepare a cherry-pick-ready patch (with a fixture test) for when the PR channel opens. This is also a good candidate for a dsh doctor-style preflight check; I can add a manifest-bom check to dsh-plugin-doctor so publish/diagnostic paths catch it before boot.

[zoahdev]

Shipped the promised preflight: dsh-plugin-doctor v1.6.0 now includes a manifest-bom check on --profile.

npx dsh-plugin-doctor --profile ~/.dsh/profiles/web --json

Verified end-to-end on Windows:

• profile package.json starting with EF BB BF ? manifest-bom: FAIL, exit 2, message names discussion Bug: profile package.json with UTF-8 BOM crashes dsh web (Unexpected token ... is not valid JSON) #1842
• clean manifest ? PASS, exit 0

Release: https://github.com/zoahdev/dsh-plugin-doctor/releases/tag/v1.6.0 ? tests 15/15 (new fixture covers BOM/clean/missing). The upstream one-line fix (readProfileManifest stripping a leading U+FEFF) is still the right permanent fix; this check makes the precondition diagnosable before boot.

[zoahdev]

Cherry-pick-ready patch is now staged on the fork, built directly on upstream master (47f9438):

• Branch: https://github.com/zoahdev/deepseek-harness/tree/fix/profile-manifest-bom-strip
• Change: readProfileManifest strips a leading \uFEFF before JSON.parse (one line, with the same fix applied to the sibling manifest reads)
• Companion preflight: dsh-plugin-doctor --profile ? --json ? manifest-bom check (v1.6.0)

When the PR channel opens, either commit can be cherry-picked as-is.

[LeonisK]

Thanks for following up on this and for putting together the fix and the preflight tool.
Your approach makes perfect sense — stripping the BOM in readProfileManifest is the cleanest way to address the root cause. The preflight check in dsh-plugin-doctor is also a great addition that will save Windows users a lot of confusion before boot.
I've reviewed your branch and the change looks solid.
Really appreciate your help on this!🙏🏻

[Electricitysheep]

UTF-8 BOM 让 JSON 解析崩（Windows 记事本保存常带 BOM）——建议：package.json 用无 BOM 的 UTF-8 保存（VS Code 默认就是无 BOM，记事本另存为选 UTF-8 不带 BOM）。

Windows 配置文件编码坑（含 BOM/换行）第 3 章有整理：https://github.com/Electricitysheep/dsh-handbook/blob/main/docs/03-profiles.md

[LeonisK]

补充：这个问题会“反复出现”，因为 profile 的 package.json 每次被编辑器/脚本重新保存时都可能再次带上 BOM。

实际遇到的情况：

• 去掉 BOM 后 dsh web 能正常启动；
• 但只要再次编辑该文件（例如修改插件依赖、调整 profile 配置），如果保存工具写了 UTF-8 BOM，启动又会立刻报同样的 is not valid JSON。

对用户的建议：

• 编辑器保存时显式选择 UTF-8（无 BOM）（VS Code：右下角编码 → 通过编码保存 → UTF-8）；
• 注意 Windows PowerShell 5.1 的 Set-Content / Out-File -Encoding utf8 会写入 BOM，尽量用 Node 或 PowerShell 7 的 utf8NoBOM；
• 自查是否带 BOM：查看文件开头 3 个字节是否为 EF BB BF。

这也说明官方侧在 readProfileManifest 里兼容 BOM（解析前 strip \uFEFF）是更稳妥的长期修复，否则每个用户都可能反复踩同一个坑。

[LeonisK]

顺便宣传一下我们的项目https://github.com/ysr666/dsh-vision-router