My testing shows this tool is not worth it? Am I doing something wrong?

[jbovard2016]

All four tools are tested. In this test, billion-context never hurt recall: every run, with or without it, scored 27/27. Whether it saves money depends on the tool you use it with:

• Fixed script: 14% cheaper. The model shrank old history into summaries when billion-context prompted it, and the provider’s discount for already-seen text mostly held.
• OMP: 12% cheaper. Same as above.
• OpenCode: 35% more expensive. Copying a session for the quiz lost the provider’s discount for already-seen text. Normal turns came out about even.
• Claude Code: 92% more expensive and twice as slow. The model ignored all three prompts to shrink its history, and billion-context reloaded the session badly between runs.
  Verdict: The repo’s big claims don’t hold up as a general claim. In this test it only paid off with OMP or a simple script, where the model actually shrank its history.

[ranxianglei]

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

查重结论: 不重复 —— 你的 Claude Code 成本/缓存发现与既有 open issue #5385(p1:长 CC 会话每轮重复缓存 ~140–175K tokens,cache read 卡在固定前缀)同症状家族,该 issue 下已有一份与你数字吻合的第二报告(v0.1.210+,cache read 逐轮冻结);对应的诊断 PR #5386(outbound 前缀 LCP diff 日志)在途。其余发现(OpenCode 拷贝成本、模型无视压缩提示、安全检查器默认放行、自更新不认关闭开关)暂无对应 issue,按政策在本线程内跟踪,不开新 issue。标签: bug / claude-code / opencode / omp / p1 / scope/wiring(含 p1 级症状与安全相关行为)。

You're not doing anything wrong — you measured exactly what the tool does, and several of your readings are right. What we found in the code for each finding:

Claude Code +92%, 2× slower — three separate causes stacking:

1. Cache instability between turns/runs. Same family as Long Claude Code session re-caches ~140-175K tokens per turn (cache read stuck at the fixed prefix) #5385. Prime suspect identified: compress.messageTimestamps, which shipped default-ON in v0.1.210 (you ran v0.1.212) — it adds a wall-clock time= attribute to every message tag. The design promise is byte-stability after first stamping, but two independent reports now suggest that promise isn't holding everywhere. Cheap discriminating test: re-run one of your lanes with "compress": {"messageTimestamps": false} in ~/.config/billion-context/billion-context.json (or env BILI_MESSAGE_TIMESTAMPS=0) and compare hit rates. If hits jump toward 95%+, root cause found.
2. "NO persisted blocks" log line. That warning prints three numbers: mapped refs, persisted blocks, compressions. When compressions=0 — your case, since the model never compressed once — raw replay is the normal shape of a never-folded session, not lost state. Share the full line (compressions=N) to confirm.
3. The model ignored all three compression prompts. That's model behavior, not a bili defect — but the consequence is real: zero token reduction while still paying all overhead (injected tools, nudges, timestamps), plus cause fix: keep system prompt byte-stable for prefix cache #1 stacked on top.

OpenCode +35% (session copy). Copying creates a NEW session id with no fold state; the first replay resends full raw history while the provider's cache holds the recently-folded view (plus possible idle TTL expiry) → one-time ~169K uncached. "Normal turns came out about even" matches this exactly. It's a design boundary (a copy doesn't inherit folds), not a per-turn leak. How did you copy the session?

Safety checker answering "allow" by default. Confirmed, and intentional: when Claude Code runs in auto mode, before every classified tool call it fires a one-shot safety classifier that RE-SENDS THE ENTIRE RAW TRANSCRIPT (O(n²) cost from day one; once the raw transcript crosses the classifier model's window the request 400s forever and every classified tool is blocked). Since #2740 (owner decision 2026-10-12), bili intercepts those calls by default and answers them locally with an allow verdict — zero egress, zero billing. So with stock config, CC's auto-mode safety layer is effectively disabled while traffic goes through bili. Off switch: BILI_AUTO_MODE_FORGE=0 (or "autoModeForge": {"enabled": false}) restores real upstream verdicts. That users discover this by surprise says the surfacing is inadequate — I'm filing a decision request as a separate comment below.

Self-updating despite auto-update off. There is no active advisory right now (the documented force-install bypass), so I need details before tracing: the exact version transition + log lines, and how you turned it off (env ACP_AUTO_UPDATE=0 vs file autoUpdate: false vs panel toggle). A manual bili update bypasses throttle/stagger by design.

OMP notes ("only works if it can edit OMP's settings file", "5 of the client's tools dropped", cost report not recognising sessions): those live on the OMP side (it owns its own settings/tools/cost report) — noted, outside bili's control.

Your overall verdict. Agreed: the headline claims don't hold as general claims at your scale. The benefit is conditional — stable client-side caching + a model that actually compresses + sessions long enough for folding to matter. Your scripted/OMP results show the intended upside; OpenCode/CC show the failure modes. I'll propose tightening the README wording (claims → conditions) to the owner; that's owner territory, so flagging rather than changing unilaterally.

What would help most (priority order):

1. One A/B run of any lane with messageTimestamps OFF (hit rate + cost) — the single most informative datapoint for Long Claude Code session re-caches ~140-175K tokens per turn (cache read stuck at the fixed prefix) #5385.
2. The full "NO persisted blocks" line (compressions=N) from your CC run.
3. CC setup: version, how it connects (native launcher vs ANTHROPIC_BASE_URL=…/bili/), whether auto mode was on.
4. For the self-update item: version transition + how auto-update was disabled + log lines.
5. Optional: re-run the CC lane with ACP_DUMP_BODY=1 — byte-level diff of consecutive requests pinpoints the unstable position (PR [PR] feat: outbound-prefix LCP triage line for prompt-cache-break diagnosis (#5385) #5386 adds opt-in prefix-diff logging behind the same dump switch).

---

结构化八问结论(中文)

1. 是什么问题:外部用户四工具对照基准显示仓库核心主张(5× token 节省、95–97% 缓存命中、全客户端透明集成)不能作为一般性结论成立;同时暴露两个具体行为——Claude Code auto-mode 安全检查被默认本地答成「放行」(设计使然但不可发现)、自更新疑似不认关闭开关(待确认细节)。
2. 什么原因导致的(分层):① CC +92% = 两个机制叠加——出站字节跨轮/跨次漂移导致前缀缓存失效(与 Long Claude Code session re-caches ~140-175K tokens per turn (cache read stuck at the fixed prefix) #5385 同家族,首要嫌疑是 v0.1.210+ 默认开启的 messageTimestamps)+ 模型无视全部压缩提示(模型行为:零压缩收益但开销照付);② OpenCode +35% = 会话拷贝得到新会话 id、无折叠状态,一次性全量重发与缓存中的折叠视图分叉(设计边界,非泄漏);③ 安全检查器默认放行 = auto mode 状态丢失，会话累计输入了593.44M tokens #2740 owner 决策的本地锻造默认开启(O(n²) 车道零计费),不是 bug 是可发现性问题;④ 自更新不认关闭 = 当前无激活 advisory,机制未定,待复现信息。
3. 影响范围与严重程度:所有经 bili 跑 CC auto-mode 的用户安全层实际被旁路(安全相关,p1 级);CC 长会话用户支付约 1.9× 成本(成本爆炸类,p1);OpenCode 拷贝场景一次性损失(p2/p3);自更新项待定。
4. 怎么修复的:安全检查项 → 保持默认(owner 原意)但加显著告知(面板横幅 + 启动日志行 + README 专节),决策请求另行提交;CC 缓存 → 收敛到 Long Claude Code session re-caches ~140-175K tokens per turn (cache read stuck at the fixed prefix) #5385,先做判别实验(messageTimestamps 关 A/B)+ PR [PR] feat: outbound-prefix LCP triage line for prompt-cache-break diagnosis (#5385) #5386 前缀 diff 日志定位漂移点,再在根因层修;OpenCode 拷贝 → 先文档化边界,折叠状态继承列为后续功能;自更新 → 信息到位后定位。
5. 是否有回归问题:本轮纯分析无代码改动,无回归;后续各修复单独过回归审查。
6. 是否破坏缓存:本轮不涉及(issue 本身即缓存失效问题)。
7. 是否改了设计方向:未改;本地锻造现方向为 owner 已决策,仅提议加强可发现性。
8. 是否引入新配置:无;现有逃生门 BILI_AUTO_MODE_FORGE=0、compress.messageTimestamps: false 均为既有配置。

判定:triage 完成,本轮不改代码;判别实验与信息请求如上,安全项决策请求见下一条评论。

[ranxianglei]

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)
🔶 需要决策

背景一句话:Claude Code auto-mode 的安全检查(每个受分类工具调用前的模型判定)目前默认被 bili 本地锻造为「放行」——#2740 的 owner 决策(该车道 O(n²) 计费且 transcript 超窗后永久 400,本地锻造实现零出站零计费);外部用户本轮靠自行测试才发现这一行为,说明安全相关的默认行为可发现性不足。

待决问题:「本地锻造」保持默认开启的同时如何告知用户,还是把默认改为透传(真实上游判定)?

选项:

• A. 保持默认 ON + 显著告知(web 面板横幅 + 启动日志一行 + README 专节 + 首次启用的一次性公告):代价 = 少量文档/UI 工作;收益 = 保住零计费与防永久 400 的原始动机,同时让安全影响在用户第一次跑 CC auto-mode 时可见。
• B. 默认改为 OFF(纯透传、真实判定):代价 = 所有 CC auto-mode 用户的 O(n²) 计费回归,transcript 超窗后车道再现 auto mode 状态丢失，会话累计输入了593.44M tokens #2740 原始故障(每次分类工具调用被永久阻塞);收益 = 安全层默认为真。
• C. 维持现状不改:代价 = 同类意外发现会再次发生,外部发现时的信任成本高;收益 = 零工作量。

推荐:A。决策原意(零计费、防超窗 400)仍然成立,但安全相关默认必须至少在使用点可见;A 不改变任何行为,只补告知。

不决定的默认行为:维持现状(默认 ON),我不擅自改默认值或动 src/auto-mode-forge.ts。

[ranxianglei]

auto model是增量车道吗?这个需要适配.ework需要整理一个全局的方案而不是打补丁

[ranxianglei]

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

一、直接回答:auto-mode 不是增量车道,也进不了增量车道。

1. 增量车道(x-bili-plugin-turn divert,Delta lane: native-mode clients send only deltas, server maintains full state + one-time fold, upstream receives snapsho | 增量车道:native 模式客户端只发增量,服务端全量维护+一次折叠,上游只收快照(#2926 的泛化) #3248/Plugin lanes: pin identity to client msgids, render outbound from the server-owned history | 插件车道服务端正史:消息身份钉在客户端 id 上,出站前缀由正史渲染 #4491)只接带宿主 shim 的插件车道(dsh/omp/opencode/pi);Claude Code 走 ANTHROPIC_BASE_URL 是代理模式,从不发这个头。
2. classifier one-shot 被拦在比车道分流更早的位置(src/server/handle.ts:2160,在 标题生成等侧请求污染主会话状态:prepareAnthropic 缺 isTitleGen 守卫、快照无条件覆盖、survivedCount 每请求空转 #388 side-request 分裂之前):默认本地锻造直接返回(不进 kernel 管道、不碰折叠状态);forge 关闭时落 side-request 透传(共享主会话 key 但不碰 kernel 状态)。
3. 形状上也不兼容增量协议:它把整份 raw jsonl transcript 摊平进 <transcript> 包装重发,带 CC 自己的 policy prompt 和严格输出契约(stop_sequences、severity 恰好一次、解析失败=fail-closed block)。增量信封预设稳定会话身份+客户端 shim 配合,CC 两者都没有。

所以「适配」≠ 把它搬上增量车道。正确方向其实已经写在 src/auto-mode-forge.ts:17 的文件头里:compressed-view swap feeding the REAL classifier——喂压缩视图给真实分类器,拿真实判定。

二、全局方案(不打补丁)。现状盘点: 辅助/side 车道今天全是逐条补丁——auto-mode 锻造(默认放行)、title-gen sidecar(#388/#2503/#2241)、dsh review blob(#4438/#1916)、dsh persona fork(KDD#9 证据白名单)、codex compaction 锻造、notification classifier 误绑主会话(#4840 open bug)……检测器、身份策略、计费策略散落在 handle.ts/prepare-anthropic.ts/session-id.ts/plugin.ts,每来一条车道加一个 if。收敛为一张辅助车道策略表,每条车道注册一个条目:

维度	声明内容
检测谓词	证据背书、偏向假阴性(漏检=原样透传,安全方向)
身份策略	共享主会话 / 独立命名空间 / 只读借用(不回写主状态)
视图策略	verbatim / ledger 渲染 / 有界压缩摘要
判定与计费	真实上游 / 本地应答(仅显式 opt-in)
降级路径	检测失败或构造失败 → 逐字节原样透传

外加一条硬规则:宿主的 safety/security 判定不得默认本地伪造——本地应答必须显式开启+强制告知。这一条把 F1 从「补个通知」变成规则级修复,以后任何新车道想默认伪造安全判定都过不了关。每条注册车道配决策矩阵正负行(#3617 纪律)。

auto-mode 作为第一条目落地:

• 检测:复用现有三段门(结构/位置/flavor,假阴性偏向,isAutoModeClassifier() false positive: once <transcript> appears in the history, compression for the entire session is | isAutoModeClassifier() 误判：历史中出现 <transcript> 后，整个会话的压缩被静默跳过 #3908 中毒教训已内置);
• 身份:绑定主会话只读——借它的 ledger/折叠状态构造视图,零回写(forge 路径现有的 metadata 计数保留);
• 视图:重建 <transcript> payload——尾部窗口逐字(pending action + 工具参数字节精确,§7.3 wire fidelity),头部折叠成紧凑摘要且保留安全相关字段(命令串、文件路径,不做散文级概括),总体积有界(与 session 长度无关)→ O(n²) 计费与超窗永久 400 两个原始病灶同时结构性消失;
• 判定:真实上游调用分类器模型,安全语义恢复;用户为此车道付费(与直连同),但压缩后每次体积近常数,长会话反而比现在 O(n²) 增长便宜;
• 降级:视图构造失败/无法绑定 → 逐字节原样透传(pre-auto mode 状态丢失，会话累计输入了593.44M tokens #2740 行为,安全方向);flavor 判不出 → 同样透传,永不发畸形判定;
• 锻造:降级为显式 opt-in(BILI_AUTO_MODE_FORGE=1),面板横幅+启动日志行+README 专节强制;
• 灰度:复用 rollout.features 百分比基建(与 pluginFullResendWatermark 同机制)逐步放量。

三、待你拍板:

• Q1 范围:全局方案 = 辅助车道策略表 + 「安全判定不默认伪造」硬规则,第一条目 auto-mode 压缩视图——这个范围对吗?
• Q2 auto-mode 默认行为:A. 真实判定(压缩视图)默认开、锻造 opt-in【推荐】;B. 维持锻造默认、真实判定 opt-in(零计费但安全层默认旁路,F1 类意外会再发生);C. 灰度期 B、终局 A(rollout 百分比 5%→100%)。推荐 C→A。代价对比:A/C = 该车道开始计费(压缩后近常数)+ 安全恢复;B = 零计费 + 安全旁路维持。
• 不决定的默认行为:不动任何代码,维持现状(锻造默认 ON),方案留本线程等拍板。

---

结构化八问(决策请求体例)

1. 是什么问题:CC auto-mode 安全检查车道目前是「默认本地伪造放行」的零计费补丁,等于默认旁路了 CC 的安全层;你已定向:要适配,要全局方案不要打补丁。
2. 什么原因导致的:两层——机制层:该车道每次全量重发 raw transcript(O(n²)、超窗即永久 400),当时选了最便宜的出口「本地答 allow」,用安全语义换可用性与成本;系统层:辅助车道没有统一策略面,每车道各打各的 if,「本地应答 vs 真实判定」这个取舍维度没有强制声明,于是「安全判定默认伪造」能静默出厂。
3. 影响范围与严重程度(按选项):A/C——所有经 bili 跑 CC auto-mode 的用户:该车道由免费变计费(压缩后体积近常数),安全层由旁路变真实(行为变更:旧=默认伪造放行 → 新=默认真实判定,PR 与发布说明必须披露);B——无行为变化,但 p1 级安全相关意外持续存在。
4. 怎么修复的(推荐项):一张辅助车道策略表(检测/身份/视图/判定/降级五维声明+矩阵行)+ auto-mode 第一条目 = 只读借用主会话 ledger 构造有界压缩 transcript(尾逐字+头摘要)喂真实分类器;锻造降为显式 opt-in+强制告知;rollout 百分比灰度。
5. 回归风险(按选项):A/C——检测漏检降级为原样透传(= pre-auto mode 状态丢失，会话累计输入了593.44M tokens #2740 状态,安全方向);视图保真风险(压缩丢掉影响安全判断的上下文)以尾窗逐字+工具参数保留缓解,并加负控制测试(已知危险动作仍被 block);isAutoModeClassifier() false positive: once <transcript> appears in the history, compression for the entire session is | isAutoModeClassifier() 误判：历史中出现 <transcript> 后，整个会话的压缩被静默跳过 #3908 类标记中毒回归由现有检测门+新矩阵行覆盖;不动 kernel/(视图构造在 src/ 复用既有导出——若发现需要新 kernel 能力,单独走 Kernel Boundary 申报)。B——无回归(现状)。
6. 是否破坏缓存:不破坏——该车道是一次性请求,不进主会话前缀缓存链;主对话出站字节不变(只读借用,零回写)。
7. 是否改了设计方向:是(正是你定向的方向调整)——辅助车道从「逐条补丁」转为「策略表+硬规则」;auto-mode 从「默认本地应答」转为「默认真实判定+显式 opt-in 本地应答」。
8. 是否引入新配置:BILI_AUTO_MODE_FORGE key 不变、默认翻转(旧语义 0=关闭锻造=真实判定,与新默认等价,已显式设置过的用户零迁移;未设置的存量用户发生本次有意的行为翻转,随发布披露)+ 一个内部 rollout key(非用户配置面)。

拍板后我按一个 PR 交付全局方案 v1(策略表骨架 + auto-mode 条目 + 默认翻转 + 告知 + 矩阵行 + 测试,引用本线程 #5843)。不依赖决策的部分已预验证完毕:主会话 ledger 只读可借(handle.ts:2160 处 session 对象已在手)、检测器可直接复用、确认无需动 kernel/。

[ranxianglei]

🤖 Powered by ework · qwen3.8-27b

(ai: vllm-qwen/qwen3.8-27b)

Addendum on the self-update item (F2) — I finished the code-side audit of v0.1.212 while waiting for details:

No code path installs a routine version with auto-update off. The 3-minute background loop only starts when the resolved setting is on (src/cli.ts:913), and even then every check re-gates on it (src/update.ts:1502). The only two paths that can still install are documented by design:

1. Advisory force-install (critical-defect blocklist only) — the list has been empty since 2026-09-27, so this cannot be what you saw recently;
2. Explicit manual actions — bili update, bili plugin update, or the web panel's 立即检查/升级 buttons (these intentionally bypass the switch).

So the most likely explanations are: (a) one of those manual actions happened during your test window and read as "self-update", or (b) the setting didn't actually take effect — note that config-file edits are not hot-reloaded: a running process keeps whatever it resolved at startup until restart (#3754). There is no autoUpdate toggle in the web panel; it's env ACP_AUTO_UPDATE=0 or file "autoUpdate": false, then restart.

Two log lines would settle it instantly:

• at startup: [update] effective settings (this process, until restart): autoUpdate=… — what the process actually ran with;
• at the moment of the update: the install log line names the path (advisory:<id> / follow-latest / manual) and the exact version transition.

(中文一行:自更新项代码侧审计完毕——v0.1.212 无「关闭后仍自动装常规版本」的路径,仅剩 advisory 强制(当前列表为空)与显式手动操作两个文档化旁路;最可能是手动操作被误读或配置未重启生效,等上面两行日志即可定案。)