Repository navigation
Releases: DDDMUC/dsh-free-search
Release list
v0.9.3
dsh-free-search v0.9.3
修复 #80:DSH 0.2.1-rc.2 把 web 的 provider 选择从 web.searchProviderId / web.fetchProviderId 属性移到了 config 的 volatile ref(web.config.searchProvider / fetchProvider,只能经 settings.mutate 写)。插件仍写旧属性,没人读它 → web_search 静默回退到官方 provider(没有 DEEPSEEK_API_KEY 时报 401)。
修了什么
- 两套 API 都兼容:有
config.*ref 时走settings.mutate("web", ops),没有时继续写旧属性 - 接管改在
ctx.inject(["settings"])里执行:apply阶段 settings 服务尚未挂载,直接取ctx.get("settings")永远是 undefined,接管会静默跳过 - fetch + search 合并成一次 mutate:同一 tick 里连发两次,第二次会被设置文档的版本号顶掉(实测 fetch 成功、search 丢失)
- 新增
tools/assert-provider-takeover.mjs(6 项),覆盖新旧两套 API、单批写入、以及"显式指定的 provider 不被抢占"
验证
- 离线断言:17 个脚本 / 501 项全绿
- 真机(0.2.1-rc.2):重启后
web.searchProvider从deepseek-official变为ddg、fetchProvider变为fs-quality-fetch;设置页 bridge 路由存在(GET 返回 405 = 需要 POST) - 旧版本 DSH(属性 API)的接管行为有单测覆盖,未改变
升级
dsh plugin --profile web add dsh-free-search@0.9.3
# 重启 dsh 生效English
Fixes #80: DSH 0.2.1-rc.2 moved web provider selection from the web.searchProviderId / web.fetchProviderId properties to volatile config refs (web.config.searchProvider / fetchProvider, writable only through settings.mutate). The plugin kept writing a property nothing reads, so web_search silently fell back to the official provider (HTTP 401 without a DEEPSEEK_API_KEY).
What changed
- Both APIs are supported: with a
config.*ref the takeover goes throughsettings.mutate("web", ops); otherwise it keeps writing the legacy property - The takeover now runs inside
ctx.inject(["settings"]): duringapplythe settings service is not mounted yet, soctx.get("settings")is always undefined and the takeover was silently skipped - fetch + search are batched into a single mutate: a second mutate in the same tick loses to the settings document revision (measured: fetch applied, search dropped)
- New
tools/assert-provider-takeover.mjs(6 checks) covering both API shapes, the single batched write, and "an explicitly selected provider is never stolen"
Verification
- Offline assertions: 17 scripts / 501 checks green
- Live on 0.2.1-rc.2: after a restart
web.searchProviderflips fromdeepseek-officialtoddgandfetchProvidertofs-quality-fetch; the settings bridge routes exist (GET answers 405 = the route needs POST) - The legacy (property API) takeover path is covered by unit checks and unchanged
Upgrade
dsh plugin --profile web add dsh-free-search@0.9.3
# restart dsh to applyv0.9.2
dsh-free-search v0.9.2
两个 PR 合并:新增 deepseek-tools 引擎,修复 Parallel MCP 卡死。感谢 @traveller1974 与 @georgeatparallel。
新增:deepseek-tools(原生搜索端点)
DeepSeek 的原生搜索接口 POST https://api.deepseek.com/v1/tools/search,与 deepseek-official 并存(不替换):
deepseek-official |
deepseek-tools |
|
|---|---|---|
| 接口 | Anthropic Messages + web_search_20250305 |
原生 /v1/tools/search |
| 每次搜索 | 一整轮模型(~5–6 s、~1100 输入 + 279 输出 token) | ~0.3 s、0 token |
| 返回 | 需解析 web_search_tool_result 块 |
现成 results[] |
- 同一个
DEEPSEEK_API_KEY,同样的设置项,无需额外配置 - 时间过滤通过 query 语法
after:实现,timeRange契约不变 - 摘要上限放宽到 2000(官方
summary600–800 字,避免被 300 字截断) - 默认回退链里排在
deepseek-official之前(同 key、快约 15×)——如果你更想保留原来的顺序,在设置页用 ↑↓ 调整即可
修复:Parallel MCP 卡死
- 25 秒超时与调用方取消现在覆盖响应体读取:此前
response.text()在try/finally之外,服务器发完响应头后卡住会一直挂住 - 解析改用共享 MCP 解析器:跳过 SSE 通知、正确拼接多行
data:帧(此前会漏掉有效结果) - 错误路径补全:非 2xx / JSON-RPC error / tool error / malformed payload 都会走既有失败链
其他
- 新增
tools/assert-parallel-mcp.mjs(15 项),并修复其在 Windows 上清理临时目录时的 EPERM - README 断言表更新为 16 个脚本 / 495 项
验证
- 离线断言:16 文件 / 495 项全绿
- 贡献者的真机验证:17 项 DeepSeek live 检查 + 一次走真实
ctx.webseam 的运行时搜索(~366 ms,明确排除 Bing HTML 回退)
English
Two PRs merged: a new deepseek-tools engine and a Parallel MCP stall fix. Thanks to @traveller1974 and @georgeatparallel.
New: deepseek-tools (native search endpoint)
DeepSeek's native search API (POST https://api.deepseek.com/v1/tools/search), alongside deepseek-official (not a replacement):
deepseek-official |
deepseek-tools |
|
|---|---|---|
| Endpoint | Anthropic Messages + web_search_20250305 |
native /v1/tools/search |
| Per search | a full model turn (~5–6 s, ~1100 in + 279 out tokens) | ~0.3 s, 0 tokens |
| Output | parse web_search_tool_result blocks |
ready-made results[] |
- Same
DEEPSEEK_API_KEY, same settings field — nothing extra to configure - Time filtering via the query syntax (
after:), thetimeRangecontract is unchanged - Snippet cap widened to 2000 (the official
summaryis 600–800 chars, so a 300 cap cut it mid-sentence) - Placed ahead of
deepseek-officialin the default fallback order (same key, ~15× faster) — reorder it in the settings UI with ↑↓ if you prefer the old order
Fix: Parallel MCP stall
- The 25 s timeout and caller abort now cover the body read:
response.text()previously sat outside thetry/finally, so a response that stalled after its headers hung forever - Results are parsed with the shared MCP parser: SSE notifications are skipped and multiline
data:frames are joined (previously valid results could be lost) - Error paths completed: non-2xx / JSON-RPC error / tool error / malformed payload all go through the existing failure chain
Other
- New
tools/assert-parallel-mcp.mjs(15 checks), and its Windows temp-dir cleanup no longer fails with EPERM - README assertion table refreshed to 16 scripts / 495 assertions
Verification
- Offline assertions: 16 files / 495 assertions green
- Contributors' live checks: 17 DeepSeek live checks plus a runtime search through the real
ctx.webseam (~366 ms, explicitly asserting it did not fall back to Bing HTML)
v0.9.1
dsh-free-search v0.9.1
解决 #69 的两个痛点:multi 档位(想快就快、想全就全)与同站刷屏(子域不再霸榜),另加一条系统提示词契约。
Multi 三档(设置页单选)
| 档位 | 并发上限 | 目标成功源数 |
|---|---|---|
Fast · 1/1 |
1 | 1 |
Balanced · 3/1 |
3 | 1 |
Deep · 6/2 |
6 | 2 |
- 并发上限是天花板,不是下限:实际并发取「上限」与「凭据合格引擎数」的较小值;凭据判定在截断之前完成,缺 key 的引擎不再白占名额。
- 目标成功源数不是硬下限:只用于触发补跑,跑不满就返回已有结果、不报错。
- 整轮 30 秒总预算(
SEARCH_BUDGET_MS):并发扇出与全部补跑共享同一个 deadline,到点即停并返回已经拿到的结果。
eTLD+1 同站判定
合并排序里的「同站」从主机名改为 eTLD+1——a.example.co.uk 与 b.example.co.uk 视为同站,域外多样性轮转在站点粒度上做,子域无法刷屏。含多段后缀表(co.uk / com.cn / github.io 等)。
系统提示词契约
明确告诉模型:检索只给候选、正文用 web_fetch,不要对每条结果都抓一遍。
文档
README 补齐 Fast/Balanced/Deep、blockedDomains、safeSearch、SearXNG Bearer Token;断言表从 6 个脚本更新为 15 个 / 463 项。
验证
- 离线断言:15 文件 / 463 项全绿
- 真机抽测:3090 极简实例(
link:源码 + 仅 free-search / delete-session 两个插件)复核档位按钮与版本号均正常
dsh-free-search v0.9.1
Addresses two #69 pain points — multi presets (fast when you want fast, thorough when you want thorough) and subdomain flooding in the merged ranking — plus a system-prompt contract.
Multi presets (radio buttons in the settings UI)
| Preset | Concurrency cap | Target sources |
|---|---|---|
Fast · 1/1 |
1 | 1 |
Balanced · 3/1 |
3 | 1 |
Deep · 6/2 |
6 | 2 |
- The concurrency cap is a ceiling, not a floor: the actual fan-out is
min(cap, credential-eligible engines). Credential eligibility is resolved before truncation, so keyless engines no longer occupy slots. - The target source count is not a hard minimum: it only triggers top-up runs; coming up short returns the results in hand instead of an error.
- A 30-second total budget (
SEARCH_BUDGET_MS) is shared by the fan-out and every top-up run; at expiry the plugin stops and returns whatever it already has.
eTLD+1 same-site detection
Merged-ranking "same site" now means eTLD+1 instead of the hostname — a.example.co.uk and b.example.co.uk count as one site, so domain-diversity rotation happens at the site level and subdomains cannot flood the results. Includes a multi-part suffix table (co.uk / com.cn / github.io, …).
System-prompt contract
The model is explicitly told that search returns candidates and web_fetch reads bodies, so it does not fetch every hit.
Docs
The README now covers Fast/Balanced/Deep, blockedDomains, safeSearch, and the SearXNG bearer token; the assertion table went from 6 scripts to 15 scripts / 463 assertions.
Verification
- Offline assertions: 15 files / 463 assertions, all green
- Spot check: the minimal 3090 instance (
link:to source, only the free-search and delete-session plugins) re-confirmed the preset buttons and the version number
v0.9.0
dsh-free-search v0.9.0
新增 4 个免 key 法律垂直引擎 和 3 个平台搜索。
法律垂直引擎(domain_search,注册表 37 → 41)
| 引擎 | 覆盖 | 接口 |
|---|---|---|
federalregister |
美国联邦法规 / 公告 / 行政令 | Federal Register API(JSON) |
courtlistener |
美国判例(11 万+) | CourtListener Search API(JSON) |
openlegaldata |
德国判例 / 法律 | openlegaldata.io(JSON,关键词走 ?text=) |
uk_legislation |
英国立法 | legislation.gov.uk Atom 源 |
全部免 key、公开接口。中国与香港的官方法律库(flk.npc.gov.cn、elegislation.gov.hk)为 SPA + 反爬,无公开 JSON API,故未接入。
平台搜索(platform_search,10 → 13)
juejin— 掘金(JSON)douban— 豆瓣(HTML 抓取)qiita— Qiita(JSON)
验证
- 离线断言:14 文件 / 434 项全绿
- 抽测:4 个法律引擎 + 3 个平台均返回结果
Added 4 no-key legal vertical engines and 3 platform searches.
Legal vertical engines (domain_search, registry 37 → 41)
| Engine | Coverage | Endpoint |
|---|---|---|
federalregister |
U.S. federal regulations / notices / executive orders | Federal Register API (JSON) |
courtlistener |
U.S. case law (110k+) | CourtListener Search API (JSON) |
openlegaldata |
German case law / laws | openlegaldata.io (JSON, keyword search via ?text=) |
uk_legislation |
UK legislation | legislation.gov.uk Atom feed |
All require no key and use public APIs. The official legal databases of China and Hong Kong (flk.npc.gov.cn, elegislation.gov.hk) are SPAs with anti-scraping measures and no public JSON API, so they were not integrated.
Platform searches (platform_search, 10 → 13)
juejin— Juejin (JSON)douban— Douban (HTML scraping)qiita— Qiita (JSON)
Verification
- Offline assertions: 14 files / 434 items all green
- Spot checks: 4 legal engines + 3 platforms all returned results
v0.8.10 - engine-retry result loss fix, ok-but-empty reporting, domain-diversity merge ranking (#69)
v0.8.10
三项修复,外加 #69 phase 1 batch 1 工作剩余的一半。
1. 引擎结果在重试路径中被静默丢弃
这是最重要的一项。runEngineTest 在首次尝试返回 0 个来源时会重试一次,但重试分支只写了一个裸的 return await attempt()。大多数引擎的 attempt() 返回 { sources, truncated },没有 ok 字段,因此检查 if (res.ok) 的调用方看到的是 undefined,把该引擎视为失败,丢弃了 sources,并报告 [unknown] (unknown error)——一个没有消息的错误。
任何首次尝试为空、重试成功的引擎,其结果都被扔掉了。在 exa 和 bing 上实际见到过。
现在两个成功路径都汇入同一个 finalize(),由它应用阻止列表、清理摘要片段,并提供 ok / truncated / content。
2. enginesUsed 统计了没有返回任何内容的引擎
只要 res.ok 为真,它就会被 push,因此返回 0 个来源的引擎仍会显示为“已使用”。更糟的是:当每个引擎都没返回任何内容时,非空的 enginesUsed 抑制了失败检查,搜索返回零结果,就好像成功了一样。
enginesUsed 现在的含义是“至少返回了一个结果”;没有返回任何内容的引擎会单独报告为 noResults: [...];全为空的搜索现在会以真实原因失败。
3. 合并排序:用域名多样性取代 seenIn(#69 §4)
合并曾按 seenInSet.size 降序排序——把“被 N 个引擎一致认可”当作正确性的证据。正如报告者警告的那样,引擎可能共享索引和排序偏差,同一域名上的六个命中并不是六个独立来源。
rankMergedResults() 现在按域名对条目分组,在组内先按新鲜度排序(仅当搜索带有时间过滤时),然后按原始顺序,再按相互印证排序;各组按每个域名的最佳条目排序,并以轮询方式交错,使前 N 个结果跨越 N 个不同域名。
只是重新排序——不会丢弃任何内容;过滤仍然是 blockedDomains 的职责。
已对照 bing 的原始结果顺序验证:原本相邻的同域条目(python.org 位于 #1/#2)现在被分开(#1 和 #7),跨引擎重复项会合并(github.com 由 bing 和 exa 都返回,只出现一次,并标记为 [seen in: bing, exa])。
4. bing:记录了为什么没有空 SERP 重试
Bing 会间歇性地向被标记的出口 IP 返回约 91KB 的空 SERP。曾尝试重试,随后移除:请求数翻倍使限流更糟,而且原因是出口 IP,不是解析。修复应放在启动器中——通过 NO_PROXY 排除 .bing.com——这正是测试中恢复 bing 的办法。
测试
assert-engines:19 -> 28(为修复 1 添加结构性防护,并防止重新加入 bing 重试)assert-result-ranking:新增,25 个断言- 测试套件:12 个文件,407 个断言,全部通过
- 版本号已在三处更新(
package.json、lib/index.js、lib/client.js)
升级
npm i dsh-free-search@latest
Three fixes, plus the remaining half of the #69 phase 1 batch 1 work.
1. Engine results were silently dropped on the retry path
The important one. runEngineTest retried once when the first attempt returned 0
sources, but the retry branch did a bare return await attempt(). Most engines'
attempt() returns { sources, truncated } with no ok field, so callers
checking if (res.ok) saw undefined, treated the engine as failed, discarded the
sources, and reported [unknown] (unknown error) - an error with no message.
Any engine that was empty on the first attempt and succeeded on the retry had its
results thrown away. Seen live with exa and bing.
Both success paths now funnel through a single finalize() that applies the
blocklist, cleans snippets, and supplies ok / truncated / content.
2. enginesUsed counted engines that returned nothing
It was pushed whenever res.ok was truthy, so an engine returning 0 sources still
appeared as "used". Worse: when every engine returned nothing, the non-empty
enginesUsed suppressed the failure check and the search returned zero results as
if it had succeeded.
enginesUsed now means "returned at least one result"; engines that returned
nothing are reported separately as noResults: [...]; an all-empty search now
fails with the real reason.
3. Merge ranking: domain diversity instead of seenIn (#69 §4)
The merge sorted by seenInSet.size descending - treating "agreed by N engines" as
evidence of correctness. As the reporter warned, engines can share indexes and
ranking biases, and six hits on one domain are not six independent sources.
rankMergedResults() now groups entries by domain, orders within a group by
freshness (only when the search carries a time filter) then original order then
corroboration, orders the groups by each domain's best entry, and interleaves
round-robin so the first N results span N distinct domains.
Reordering only - nothing is discarded; filtering stays the job of blockedDomains.
Verified against bing's raw result order: same-domain entries that were adjacent
(python.org at #1/#2) come out separated (#1 and #7), and cross-engine duplicates
collapse (github.com returned by both bing and exa appears once, marked
[seen in: bing, exa]).
4. bing: documented why there is no empty-SERP retry
Bing intermittently serves a ~91KB empty SERP to flagged egress IPs. A retry was
tried and then removed: doubling the request count made throttling worse, and the
cause is the egress IP, not parsing. The fix belongs in the launcher - NO_PROXY
excluding .bing.com - which is what restored bing in testing.
Tests
assert-engines: 19 -> 28 (structural guards for fix 1, and against re-adding the
bing retry)assert-result-ranking: new, 25 assertions- Suite: 12 files, 407 assertions, all green
- Version bumped in all three places (
package.json,lib/index.js,lib/client.js)
Upgrading
npm i dsh-free-search@latest
v0.8.9 - research call* wrappers restored - #75
Issue #75 - research: four call* wrappers were called but never defined
research failed 100% of the time whenever SILICONFLOW_RERANK_API_KEY was configured — the one step the README and PR #59 recommend.
Root cause
Four wrappers are called nine times in lib/research.js and were never defined, never imported:
| Wrapper | Call sites |
|---|---|
callJudgeQueryIntent |
193 |
callDecide |
295, 364 |
callEmbed |
915, 1016 |
callRerank |
930, 1044, 1195, 1224 |
Line 8 imports the underlying implementations (rerank, embed, decide, judgeQueryIntent) and line 13 declares CONTROLLED_STUBS — but the pass-through layer between them did not exist. Sibling helpers in the same file (judgeIntentWithDecision, askNoulSufficiency, pickRewrite, compactWindowRep) are all defined; only these four were missing.
Why it hid for six releases
Every call site sits behind a state.rerankKey / state.decisionKey guard. Without the key, research looked healthy — the notes merely recorded notEvaluated: stage-A rerank unavailable, and rerank and embedding never actually ran for anyone. With the key set, the embed branch activated and threw ReferenceError: callEmbed is not defined on the first call, taking the whole tool down.
This shipped in v0.8.0 (4635e19, PR #59) — the wrappers were absent from the commit that introduced research, not removed by a later refactor. Verified across v0.8.0–v0.8.8: nine call sites, zero definitions in every tag.
Fix
Four pass-through wrappers restored after CONTROLLED_STUBS, following the design already established there — stub first, otherwise the underlying implementation:
async function callRerank(query, documents, apiKey) {
return CONTROLLED_STUBS.rerank ? CONTROLLED_STUBS.rerank(query, documents, apiKey) : rerank(query, documents, apiKey);
}Same shape for callEmbed(texts, apiKey), callDecide(state, questions, apiKey) and callJudgeQueryIntent(question, apiKey).
callDecide and callJudgeQueryIntent are restored too, not just the two that threw first: DECISION_MODEL_URL + DECISION_MODEL_API_KEY are unconfigured today, but enabling them would have failed identically.
Tests
tools/assert-research-call-wrappers.mjs (23 assertions), zero dependencies like the rest of tools/:
- static — every called
call*has a definition, has call sites (not dead code), takesapiKeylast, prefers the stub and falls back to the implementation - behavioural — the wrappers are extracted from the shipped source and invoked for real, with the underlying implementations replaced by throwing sentinels: a wrapper that bypassed the stub would blow up
Pointed at the pre-fix research.js it fails 18 of 23.
End-to-end A/B on macOS with Edge 155, same host, same credentials, same prompt, only the file swapped: the pre-fix file threw Error: callEmbed is not defined on every call; the fixed file completes and returns a brief.
Credits
Reported by @Phi-2X, with a verified patch and a full root-cause write-up — the call-site inventory, the masking analysis and the signature cross-check were all correct.
Version strings realigned across package.json, lib/index.js and lib/client.js.
v0.8.8 - macOS browser-tier teardown fix - #74
Issue #74 - macOS: uncaught spawn taskkill ENOENT killed the DSH host
The T1.5 system-browser tier tore down with an unconditional spawn("taskkill", …). taskkill is a Windows-only executable, while BROWSER_CANDIDATES deliberately lists the macOS Chrome/Edge paths — so the tier is reachable by design on macOS, and every teardown spawned a command that does not exist there. spawn() returns synchronously and delivers ENOENT as an asynchronous 'error' event, which the surrounding try/catch can never see; with no listener attached Node rethrows it as an uncaughtException and the whole DSH host dies mid-fetch.
Teardown
| Platform | How the tree is killed |
|---|---|
| Windows | taskkill /pid <pid> /T /F — command byte-identical to before |
| non-Windows | process.kill(-child.pid, "SIGKILL") on the browser's own process group |
The POSIX branch needed one further change: the browser is now spawned detached, so it leads its own process group and the group kill is actually meaningful. Measured on Edge 155 with 15 live processes, child.kill() alone leaves 1 renderer behind; the group kill leaves 0. detached: process.platform !== "win32" keeps Windows spawn semantics untouched.
Error handling
The spawn() result now carries on("error"), degrading to child.kill("SIGKILL") instead of taking the host down, plus unref() so a pending teardown never holds the event loop. Cleanup paths that can fail now fail harmlessly: taskkill missing the process, the browser already exited, the pid already reaped.
Tests
tools/assert-killtree-portable.mjs (8 assertions): the platform guard and the async error listener must exist in the shipped source, and the shipped killTree is executed against a real detached process group to assert no uncaughtException and no survivors. Mutation-tested — dropping detached, dropping on("error"), or degrading the group kill back to child.kill() each turn it red. Against the pre-fix file it fails 6 of 8, including the behavioural checks.
The win32 branch is covered separately by forcing process.platform = "win32" and driving the spawn failure paths (async ENOENT, synchronous throw, normal start): all degrade correctly without touching the host.
Version strings realigned across package.json, lib/index.js and lib/client.js, which had drifted to 0.8.6 while the registry was at 0.8.7.
Not touched
dshmarket/lib/dsh-cli.js and @zilliz/memsearch-dsh/index.js carry the same unguarded pattern, but they are separate packages and were left alone. Worth extracting a shared helper.
Credits
Reported by @dotdance — the root cause was exact, and the suggested fix was behaviour-identical on Windows, which is what shipped here.
v0.8.7 - Domain blocklist (blockedDomains) - #72
Issue #72 - P1 filtering + P2 settings UI
Domain blocklist: keep content-farm sites out of search results before they reach the model.
P1 - filtering
New \lib/domain_filter.js:
- \isDomainBlocked\ - exact domain or subdomain match (\csdn.net\ covers \�log.csdn.net), case/w-\www.\ normalisation, no regex (
otcsdn.net\ is never hit) - \setActiveBlockedDomains\ / \getActiveBlockedDomains\ - per-request active list
- \�pplyBlockedFilterToResult\ - result-level choke point
ormalizeDomainList\ - strips scheme/path/port/\www., lowercases, dedupes
The filter is applied at every point where a result leaves the engine layer, not inside \uniqueSources:
| Choke point | Covers |
|---|---|
| \provider.search\ engine result, before the cache write | all engines |
| \provider.search\ cache hit | stale entries written before the fix |
| \ | |
| unEngineTest\ success return | multi / engine test / settings bridge |
| \ | |
| unMultiSearch\ merge loop | the \multi_search\ tool path |
| \platform_search\ tool | the only path with no prior coverage |
\uniqueSources\ still filters before dedupe and \maxResults, so a blocked entry yields its slot to the next candidate and the result count does not shrink.
\�uildCacheKey\ includes the normalised blocklist, so changing the config immediately invalidates stale cache entries.
esearch.js\ filters cluster candidates and the \etchTiered\ entry. An empty list means no filtering at all - zero behaviour change.
P2 - settings UI
Free-form multi-line textarea (one domain per line) in the Free Search settings page, placed after the fallback triggers. Non-blocking hints report auto-corrected, duplicate and unrecognized lines; blank lines are ignored silently. Full URLs can be pasted directly. .dshfs-textarea\ reuses the .dshfs-input\ visual language.
A cross-check assertion extracts \parseBlockedDomains\ from the client and compares its output against the server-side
ormalizeDomainList\ on the same inputs, so the two normalisers cannot drift.
Fix during the cycle
The first P1 cut anchored the filter inside \uniqueSources, which only 29 of 40+ engines run through. \searchAnysearch, \searchPlatform\ and 9 site engines return \sources\ directly, and the \multi_search\ tool calls
unMultiSearch\ without going through \provider.search\ - so the active blocklist was never set on that path. Live testing surfaced this; the filter now lives at the result exits instead of a shared helper.
Also: uniform row height in the engine priority list (keyless rows no longer collapse).
Tests
8 assertion suites, all green. \�ssert-blocked-domains\ grew 29 -> 69 checks, adding result-choke-point semantics,
ormalizeDomainList, and the client/server consistency check.
v0.8.6
dsh-free-search v0.8.6
修复发布。修复 research 抓取链、Bing 搜索与 weather 工具契约(issue #70 / #71 / #73)。
修复
- research 抓不到正文(#70)—
platformFetch把 AbortSignal 包成了对象({ signal })传给宿主ctx.web.fetch,导致直连层抛 TypeError、浏览器兜底层也随之失效;现改为直接传 signal 本体。同时parseUpdatedFromHtml一直只有调用没有定义(抓取成功也会抛 ReferenceError),现已实现(article:modified_time/og:updated_time/ JSON-LDdateModified/<time datetime>)。 - Bing 0 结果 + 跳转链接(#71)—
mkt=zh-CN默认值在中国大陆以外会返回空 SERP 容器,现默认不再发送mkt(仅在设置页显式选择时固定);Bing 的ck/a跳转包装(真实地址在u参数,base64url)现已解包,引用与 web_fetch 拿到的都是真实 URL。 - weather 工具 100% 失败(#73)— output schema 漏声明
latitude/longitude/timezone/current.code/daily[].code,与additionalProperties: false冲突。现已补齐,并新增sanitizeWeatherResult()裁剪 undefined 字段。**新增tools/assert-weather-contract.mjs(11 项)**从两个方向锁定契约:返回值不得超出 schema、schema 声明的字段必须都有。
改进
- 浏览器兜底层(T1.5)加固:独立
--user-data-dir(避免与用户正在运行的浏览器冲突)、taskkill /T整树清理、stderr 读取丢弃、exit+flush 语义、禁用崩溃上报/同步;同源 Referer 经实测无收益已回退。
测试
贡献者套件 95 项全部通过(含新增 weather 契约 11 项);端到端实测:weather 中英文输出、video_search 播放链接、research 正文抓取、Bing 直链解包均正常。
致谢
- @jasper552 — #70:精确定位
{ signal }与未定义的parseUpdatedFromHtml - @pablosilva87 — #71:完整的根因矩阵(mkt × headers)与
ck/a解包方案 - @vrvtvy — #73:指出 weather 的 schema 契约缺口与其它工具的裁剪惯例
v0.8.5
dsh-free-search v0.8.5
抓取增强发布。直连补齐真实浏览器指纹;直连被拦时改用系统浏览器渲染兜底(issue #70 的后续)。
新增
- T1.5 系统浏览器抓取层 — 直连失败(反爬/JS 挑战)时,自动用系统 Edge / Chrome 无头渲染(
--dump-dom)获取最终页面;找不到浏览器、超时或挑战未过则静默跳过,不改变原有行为。普通站点仍走直连(零额外开销)。混合样本实测成功率 5/8 → 6/8(百度百科等可被救回,耗时约 5-8 秒/页)。 - 完整浏览器请求头 — 直连补齐
sec-ch-ua、Sec-Fetch-*、Upgrade-Insecure-Requests、完整Accept、Accept-Encoding(含 gzip/br/deflate 解压),并附带同源Referer。
说明
- 知乎 / CSDN 等对自动化环境(含无头浏览器)有专门检测,匿名抓取无法通过;此类站点建议配置 Exa / Firecrawl 的免费额度 key,由专业抓取服务处理。
测试
贡献者套件全部通过(assert-engines 19、assert-research-degradation 19、assert-fetch-chain 8、assert-vertical 8、assert-free-search-test-schema 8、assert-single-fetch-layer 22)。
致谢
- @jasper552 — 报告 #70(视频搜索失效、research 只给摘要)。
Fetching Enhancement Release. Direct fetching now fills in realistic browser fingerprint details; when direct fetching is blocked, it switches to system browser rendering as a fallback (follow-up to issue #70).
Added
- T1.5 System Browser Fetching Layer — When direct fetching fails (anti-scraping/JS challenge), automatically use system Edge / Chrome headless rendering (
--dump-dom) to get the final page; if no browser is found, it times out, or the challenge is not passed, silently skip without changing existing behavior. Normal sites still use direct fetching (zero additional overhead). Measured success rate on mixed samples: 5/8 → 6/8 (Baidu Baike and similar sites can be rescued; takes about 5–8 seconds/page). - Full browser request headers — Direct fetching now fills in
sec-ch-ua,Sec-Fetch-*,Upgrade-Insecure-Requests, fullAccept,Accept-Encoding(including gzip/br/deflate decompression), and includes a same-originReferer.
Notes
- Zhihu / CSDN and similar sites have dedicated detection for automated environments (including headless browsers), and anonymous scraping cannot pass. For such sites, it is recommended to configure Exa / Firecrawl free-tier keys and let professional scraping services handle them.
Tests
All contributor suites pass (assert-engines 19, assert-research-degradation 19, assert-fetch-chain 8, assert-vertical 8, assert-free-search-test-schema 8, assert-single-fetch-layer 22).
Thanks
- @jasper552 — reported #70 (video search broken, research only gives summaries).