Repository navigation
Replies: 2 comments
Evidence chain — why a turn/step-null replace marker permanently breaks the token meter (and thus /compact)EN / EnglishSix steps, each with source Step 1 — the meter folds the entire durable log, no way to skip a bad event
while (state.consumedEvents < session.events.length) {
const event = session.events[state.consumedEvents];
this._foldEvent(session, state, event);
state.consumedEvents += 1;
}
Step 2 — every
|
Fix proposals — making the token meter and foldSurface agree on turn/step-null replace markersEN / EnglishThree options, each with ✅/ Option 1 — upstream: meter skips step-pairing for turn-null replace events (recommended)In if (event.type === "assistant/message") {
const isMarker = event.surfaceOp?.op === "replace" && event.data?.turn == null && event.data?.step == null;
if (!isMarker) { /* existing step-pairing check */ }
// else: fold surface tokens only, no step requirement
}
Option 2 — plugin-side: write markers inside an open stepChange marker emission so a marker carries the
Option 3 — plugin-side: pre-compact marker cleanup (
|
Uh oh!
There was an error while loading. Please reload this page.
EN / English
Found earlier in our daily usage; re-verified against the current npm line (through
0.1.3-alpha.2) before posting — still present. We publish one topic per report so each can be tracked independently; more findings will follow in the same form.Authorship note: reproduced and documented by me in real-world usage; root-cause tracing and drafting assisted by AI; verification and publication by me. I have no engineering background — if any technical claim reads wrong, please call it out; I will re-verify against the toolchain and correct.
Summary
/compact(manual compaction) fails hard on sessions that contain a plugin-authoredassistant/messagemarker withturn: null, step: null(a surfacereplaceevent, the same primitive built-in compaction uses). Error:token meter: assistant/message at seq N has no matching step/start event. The event is fully legal for the persistence read path (foldSurfaceaccepts it), so the session loads and works — but every pressure measurement and manual compaction after that point throws. We hit this on 7 sessions across our workspace;/compactstayed broken until the marker was removed.This is another instance of "one log, two consumers, two verdicts" (see #3632):
foldSurface(persistence) and the token meter (measurement) disagree about the same event.Trigger Condition
A session log contains a
surfaceOp: replaceassistant/messageevent whosedata.turnanddata.steparenull, written outside of any open step (e.g. betweenturn/endand the nextturn/start). Then:dsh session compact/ the/compactcommand → throwstoken meter: assistant/message at seq N has no matching step/start event;Minimal Repro
{"type":"assistant/message","seq":N,"time":T,"data":{"turn":null,"step":null,"message":{...}}, "sourceEventSeqs":[...],"surfaceOp":{"op":"replace","start":A,"end":B}}/compact.command/donecarrieskind:"error", text:"token meter: assistant/message at seq N has no matching step/start event"; no checkpoint is written./compactworks again.Analysis (root cause)
dsh-token-meter/lib/index.js:590(v0.1.1-rc.2; →:727on 0.1.2-rc.1, →:753on 0.1.3-alpha.2, verbatim) requires everyassistant/messageto match the currently open step:The meter folds the whole durable log (
_syncconsumessession.eventsfrom seq 0;index.js:556), so a single marker written between turns (no open step →stepStart === void 0) makes every subsequentmeasure()call throw. The persistence layer, by contrast, validates the same event as legal:foldSurfaceonly requires the replace'sstart/endto exist in the surface andsourceEventSeqsto cover the shadowed nodes (dsh-session/lib/index.jsassertProvenance/planSurfaceEvent). One consumer accepts, the other rejects.Compaction depends on the meter:
dsh-compaction-basiccallsctx.tokenMeterfor pressure and for re-pricing after the replacement. When the meter throws, the compaction transaction fails before any checkpoint is written (the log recordscompaction/start…command/done:error, surface unchanged). Manual/compactreportssummary/commit-class failures; automatic step-boundary pressure checks merely log a warning and continue (as the compaction README states), which is why sessions keep working for normal use while/compactstays permanently broken.Attribution
turn/step = nullfor boundary markers, which is M1-legal for the persistence layer ("null turn/step assistant/message must be replace, never append") but invisible to the meter's step-pairing rule. We own this design debt; the ecosystem-level fix belongs upstream (see proposal).Proposed fix
dsh-token-meter._foldEvent, skip the step-pairing check forassistant/messageevents whosedata.turn/stepare null and whosesurfaceOpis areplace(the marker shape foldSurface already legitimizes). Fold their surface tokens for pricing; do not require an open step. One-line guard; keeps the meter strict for real assistant messages.dsh-log-contract checknow includes a token-meter fold (rule T1) so sessions that would fail/compactare flagged before the user hits it.Related
Environment
dsh-compaction-basic(default),dsh-command-compactmountedReported by the OfferKuai Team — Founder: Zhaofeng (Yaming). Website: https://www.offerkuai.com/ | Contact: contact@offerkuai.com
中文版 / ZH
此为我们在日常使用中较早发现的问题,发布前已对照 npm 最新线(至 0.1.3-alpha.2)复核,确认仍存在。我们按"一帖一主题"发布以便独立跟踪,后续发现也将以同样形式陆续发出。
协作说明:本文由我在真实使用中复现并记录;根因与成文由 AI 协助完成,我负责校验与发布。我非工程背景,技术表述如有错误,欢迎指出——我会回去用工具链重新验证后更正。
摘要
凡会话日志中包含插件写入的、
turn/step为 null 的assistant/messagemarker(surfaceOp: replace,与内置压缩同原语),该会话的/compact与手动压力测量必然失败,报token meter: assistant/message at seq N has no matching step/start event。该事件对持久化读路径完全合法(foldSurface接受),会话能正常加载使用——但从该 marker 起,每一次压力测量/手动压缩都抛错。我们在工作区 7 个会话上复现;移除 marker 前/compact一直不可用。这又是「one log, two consumers, two verdicts」(见 #3632)的一个实例:
foldSurface(持久化)与 token meter(计量)对同一合法事件结论相反。触发条件
会话日志包含一条
surfaceOp: replace的assistant/message,其data.turn与data.step为 null,且写在任何打开的 step 之外(如turn/end与下一个turn/start之间)。此后:/compact与手动压力检查均抛同一错误。最小复现
{"type":"assistant/message","seq":N,"time":T,"data":{"turn":null,"step":null,"message":{...}}, "sourceEventSeqs":[...],"surfaceOp":{"op":"replace","start":A,"end":B}}/compact。command/done携带kind:"error", text:"token meter: assistant/message at seq N has no matching step/start event",不产生检查点。/compact恢复可用。具体分析(根因)
dsh-token-meter/lib/index.js:590(v0.1.1-rc.2;0.1.2-rc.1 移至:727、0.1.3-alpha.2 移至:753——同一 throw 逐字保留)要求每条assistant/message匹配当前打开的 step:meter 折叠整份持久化日志(
_sync从 seq 0 消费session.events,index.js:556),因此任何一条写在轮次之间的 marker(无打开 step →stepStart === void 0)都会让此后的每次measure()抛错。而持久化层判定同一事件合法:foldSurface只要求 replace 的start/end存在于 surface、sourceEventSeqs覆盖被遮蔽节点(dsh-session/lib/index.js的assertProvenance/planSurfaceEvent)。一个消费者接受,另一个拒绝。压缩依赖 meter:
dsh-compaction-basic用ctx.tokenMeter做压力判断与替换后重新计价。meter 抛错 → 压缩事务在任何检查点写入前失败(日志只留compaction/start…command/done:error,surface 不变)。手动/compact报 summary/commit 类失败;自动 step 边界压力检查记 warning 后继续(dsh-compaction-basic/README.zh.md),所以会话日常使用正常,唯独/compact永久失效。归因
turn/step=null——对持久化层 M1 合法("null turn/step 的 assistant/message 只能 replace 不能 append"),但 meter 的 step 配对规则看不见它。这是我们的设计债;生态级修复应在上游(见建议)。建议
dsh-token-meter._foldEvent对data.turn/step为 null 且surfaceOp为replace的事件(foldSurface 已认可的 marker 形态)跳过 step 配对检查,仅按其 surface tokens 计价。一行守卫;对真实 assistant 消息仍保持严格。dsh-log-contract check已加入 token-meter 折叠(规则 T1),在用户触发/compact前就标记出会失败的会话。相关
本报告由 OfferKuai(Offer快)团队提交 —— 创始人:Zhaofeng(Yaming)。官网:https://www.offerkuai.com/ | 联系:contact@offerkuai.com
All reactions