dsh-session-log-deepseek attaches the full session-log suffix to every official request with no size cap; after a format-version change the acceptance watermark can never be written, permanently bricking the session with HTTP 413
#6847
Replies: 3 comments
摘要
该字段挂在每一个官方请求上（普通对话和 compression 都走同一条路径）， 更重要的一条原则：遥测/同步类扩展绝不应该让用户的请求失败。 这里它做到了。 环境
现象在 Web UI 里对该会话发送任何消息（含 返回体是网关页而非模型错误（openresty 的 413 页）。 证据 1 · 插桩实测真实请求体把
167.50 MB 那次请求的顶层字段拆解： ⇒ 对话正文只有 0.19–1.36 MB，而请求 167–168.87 MB，其中约 167 MB 全在 证据 2 · 独立交叉验证（不用代理，直接重解析落盘日志）按插件自身的取后缀规则，重解析磁盘上的 canonical log： 与 HTTP 实测同一量级（差异来自 证据 3 · 413 来自网关，400 来自模型同一端点、同一 key 实测：
⇒ 二者无关。大会话本身不会失败，只有这个日志字段会。 根因（精确代码位置）
// 行 86：没有匹配水位时，游标从 -1 开始
let throughSeq = previous?.throughSeq ?? -1;
// 行 95：水位必须与「当前会话格式代」一致，否则跳过
if (acceptedFormatVersion !== session.header.version) continue;
// 行 126：后缀 = 从 afterSeq+1 到末尾的全部事件 —— 无上限、无分块
const suffix = session.snapshotEvents(SessionLogOffset(afterSeq + 1));
// 行 136：只有被接受之后才追加水位
accept: () => { session.append("session-log-deepseek/delivery-accepted", {
sessionId: session.id, sessionFormatVersion: session.header.version, throughSeq }); }
// 行 823-826：扩展字段被无条件并入 payload
payload: JSON.stringify({ ...body, ...extensions.fields })
// 行 1338：非 2xx 分支……（行 1355 throw）
if (!response.ok) { ... }
// 行 1362：……所以 accept() 只在 2xx 时可达
await extensions.accept();（另见行 1321 / 2745：普通消息与 compression 都走 为什么不可自愈触发条件（任一即可）：
影响面
临时规避（已验证）# ~/.dsh/profiles/web/cordis.patch.yml
- id: session-log-deepseek
config:
enabled: false
建议修复（按优先级）
一个我们未能确定的观察（请维护者确认，非断言）在这台机器的 全部 78 份 session-log artifact 中， 我们无法确定这属于哪种情况：
补充：该包自 判定方法（必须按 JSON 顶层 zstd -dc session*.jsonl.zstd | grep -c '"type":"session-log-deepseek/delivery-accepted"'最小复现步骤
已脱敏不含会话正文、API key、内部网络地址或个人信息；会话 id 以 |
附录 A · 全机 artifact 扫描（前 10，按后缀体积排序）注意：每一份 artifact 的匹配水位都是 附录 B · 扫描脚本（供维护者/其他用户自查）#!/usr/bin/env python3
"""扫描 dsh 会话日志，判断 session-log-deepseek 的「水位是否缺失 + 后缀有多大」。
用法：python3 dsh-watermark-scan.py
（只读：仅读取 ~/.dsh/sessions 下的日志并统计，不联网、不修改任何文件。）
需要：pip install zstandard
输出：每份 artifact 的格式代 / 水位 / 匹配数 / 事件数 / 后缀字节 / 全量字节，
并标出「水位=0 且后缀巨大」的同型炸弹。
"""
import json, os
import zstandard
ROOT = os.path.expanduser("~/.dsh/sessions")
def scan(path):
dctx = zstandard.ZstdDecompressor()
with open(path, "rb") as fh, dctx.stream_reader(fh) as r:
text = r.read().decode("utf-8", "replace")
lines = text.split("\n")
if lines and lines[-1] == "":
lines.pop()
if not lines:
return None
hdr = json.loads(lines[0])
ver, sid = hdr.get("version", 0), hdr.get("id")
total_bytes = sum(len(l) for l in lines) + len(lines) - 1
matched, matched_any, after_seq, n_events, suffix_bytes = 0, 0, -1, 0, 0
evs = []
for l in lines[1:]:
try:
e = json.loads(l)
except Exception:
continue
evs.append((len(l), e))
n_events += 1
if e.get("type") == "session-log-deepseek/delivery-accepted":
d = e.get("data") or {}
matched_any += 1
if d.get("sessionFormatVersion", 0) == ver and d.get("sessionId") == sid:
matched += 1
t = d.get("throughSeq")
if isinstance(t, int) and t > after_seq:
after_seq = t
for ln, e in evs:
if e.get("seq") is not None and e["seq"] >= after_seq + 1:
suffix_bytes += ln + 1
return dict(ver=ver, sid=sid, total=total_bytes, events=n_events,
acc_any=matched_any, acc_match=matched, after=after_seq,
suffix=suffix_bytes, path=path)
rows = []
for dirpath, _, files in os.walk(ROOT):
for fn in files:
if fn.startswith("session") and fn.endswith(".jsonl.zstd"):
p = os.path.join(dirpath, fn)
try:
r = scan(p)
if r:
rows.append(r)
except Exception as ex:
rows.append(dict(path=p, err=str(ex)[:60]))
rows.sort(key=lambda r: r.get("suffix", 0), reverse=True)
print(f" {'fmt':<5}{'wm':>8}{'match':>6}{'events':>8}{'suffix':>12}{'total':>11} session")
bad = 0
for r in rows:
if "err" in r:
print(f" [read failed] {r['err']} {r['path']}")
continue
flag = ""
if r["acc_match"] == 0 and r["suffix"] > 10 * 1048576:
flag = " <-- BOMB (wm=0 + huge suffix)"
bad += 1
print(f" v{r['ver']:<4}{r['after']:>8}{r['acc_match']:>6}{r['events']:>8}"
f"{r['suffix']/1048576:>11.2f}M{r['total']/1048576:>10.2f}M {(r['sid'] or '?')[:8]}{flag}")
print(f"\n scanned {len([r for r in rows if 'err' not in r])} artifacts; {bad} bomb(s) of this shape")附录 C · 与本次升级相关的其他 breaking change（仅供参考，不属于本议题）
|
|
Independent reproduction on macOS — confirms the mechanism here, with a different gateway failure mode (TRANSPORT / 550 rather than 413) and one refinement to the trigger condition. Environment: dsh CaseSession Each attempt takes a very consistent 36–38 s and yields no HTTP response at all — except roughly once per retry storm, where the gateway answers Watermark checkIdentical to the table in #6862 — zero
Refinement: the trigger is not only a format-version changeHere nothing was migrated or re-versioned. The sessions simply predate the plugin: Ruled outSame machine, same key, same minute: Workaround verified live, no restart neededAdding this to - id: session-log-deepseek
config:
enabled: falseThe +1 to the principle. A telemetry/sync extension must not be able to fail the user's request; a size cap (skip the field above N bytes) would have kept all three of these sessions alive. |
Uh oh!
There was an error while loading. Please reload this page.
Summary
@deepseek-ai/dsh-session-log-deepseek(enabled by default) attaches the entire sessionevent suffix to every official API request via the
dsh_session_logfield. When noacceptance watermark matching the current session format generation exists, the cursor starts
at
-1, so the suffix is every event from seq 0 — with no size cap and no chunking.For a long session this makes a single request body >160 MB, which the gateway rejects with
413. Because the acceptance watermark is only appended after a 2xx response, the suffix can
never shrink: permanent deadlock, and the user-visible symptom is that the session can no
longer send any message.
The field rides on every official request (normal turns and compaction), so switching
the summarization model to a local one does not work around it. Disabling the plugin is
currently the only escape.
The broader principle: a telemetry/sync extension must never be able to fail the user's
request. Here it can, and does.
Environment
0.1.6-alpha.1(npmlatest=0.1.5-rc.1;0.1.6-alpha.1is the newest published)@deepseek-ai/dsh-session-log-deepseek0.1.6-alpha.1(package has existed since0.1.2-alpha.2)Symptom
Sending any message to that session in the web UI (including
/compact) fails with:The response body is a gateway page, not a model error:
Evidence 1 — instrumented measurement of the actual request
We temporarily pointed
llm-deepseek.baseURLat a local logging proxy to measure the realpayload instead of guessing. Raw log (same session, same model, same
max_tokens):dsh_session_logpresentTop-level field sizes for the 167.50 MB request (proxy breakdown):
⇒ The conversation is ~0.19–1.36 MB. ~167 MB of a 168.87 MB request is the
dsh_session_logfield. Removing it takes the same request to 1.59 MB → HTTP 200(a ~106× reduction in payload; ~760× for the small-request variant).
Each failed attempt still uploads the full 160+ MB before failing (15.8–17.3 s per attempt),
with no warning, no cap and no backoff.
Evidence 2 — independent cross-check by re-parsing the canonical log
Without the proxy, we re-parsed the on-disk canonical log and computed the suffix by the
plugin's own rule. Both artifacts of the affected session:
Same order of magnitude as the wire measurement (differences come from
wireEventfieldrewriting and the log continuing to grow). A scan of all 78 local artifacts is in the appendix.
Evidence 3 — 413 is the gateway, 400 is the model
Same endpoint, same key:
max_tokens256kThis model's maximum context length is 1048576 tokens...⇒ The two are unrelated. A large session does not by itself fail: only the log field does.
Root cause (exact code)
@deepseek-ai/dsh-session-log-deepseek/lib/index.js:@deepseek-ai/dsh-llm-deepseek/lib/index.js:(Also
prepareRequestExtensionscalls at lines 1321 and 2745: normal messages and compactionboth go through the same path.)
This ordering is what makes the failure permanent: acceptance requires a 2xx; a 2xx requires
the suffix to fit; the suffix only ever grows.
Why it cannot self-heal
Trigger conditions (either is sufficient):
was just enabled, or enabled for the first time on an old session) starts at
-1.Existing users with long sessions are bricked as soon as the default-on plugin applies.
session id and format generation, so after a migration every old watermark stops
matching and the cursor resets to
-1.Impact
ordinary chat turns.
request-construction layer, independently of which model summarizes.
latency, no warning, no cap, no backoff.
Workaround (verified)
Disable the plugin in the profile overlay (the path documented in the dsh README):
Documentation gotcha worth noting: putting the same under
~/.dsh/settings.yamlhas noeffect (verified: after restart the field is still injected and the request is still 167 MB).
Only the profile
cordis.patch.ymlworks. It would help to make that distinction explicit inthe README.
Suggested fixes (in priority order)
extension payload (especially 413), retry once without the field rather than failing the
turn. This single change removes the whole class of failure.
window (or nothing) and record "skipped" in the watermark semantics — never an unbounded
resend from seq 0.
acceptance record for the new generation, or define "no watermark" as "start from a bounded
window" instead of seq 0.
"160 MB upload + 16 s timeout" cannot repeat silently.
An observation we could not resolve (please confirm — this is not a claim)
Across all 78 local session-log artifacts on this machine, the number of
session-log-deepseek/delivery-acceptedevents is exactly 0 — and in fact the number ofsession-log-deepseek/*events of any type is 0.We do not know which of these is true:
semantics never hold, every request uploads the full log, and issue 1 above is far more
severe than the file-size case alone); or
active, so there was simply nothing to record.
We note that the package itself has been published since
0.1.2-alpha.2, so "the plugin isbrand new" does not by itself explain the zero.
Method (JSON top-level
typeonly — a plain string grep is unreliable because session contentcan contain the same literal):
Minimal reproduction
dsh0.1.6-alpha.1 withsession-log-deepseekat its default (enabled).session: 58,693 events / ~145 MB suffix).
dsh_session_logsuffix → 413; retrying does notconverge (the watermark is never written).
enabled: falsein the profile overlay → the same session immediately returns200.
Redaction
This report contains no conversation content, API keys, internal addresses or personal data.
Session ids are abbreviated (
session-6c0de4b8…).All reactions