[dsh_session_log] First upload carries the entire session log, permanently breaking long sessions with HTTP 413

[bozhang1214]

Environment

• dsh 0.1.6-alpha.1 (profile: web)
• Provider: deepseek-official, wire protocol messages (the new default since b0641b83fc)
• Model: deepseek-v4-pro / deepseek-flash (reproduces on both)
• session-log-deepseek enabled with defaults (2389b65246 made the upload default-on)
• macOS, Node v24.15.0, pnpm dsh web

Symptom

Long-lived sessions fail every turn with:

{"message":"DeepSeek Messages request failed (413)","code":"INVALID_REQUEST","status":413}

recorded in the session log as assistant/attempt → turn/end. A brand-new session
works fine with the same provider, model, and settings. The failure is permanent:
the session never recovers, no matter how many times it is retried or restarted.

The 413 is returned by the provider, not by the local dsh web server. The local
HTTP bridge caps buffered bodies at 300 MiB and streams responses, so it is not the
source.

Evidence

Four sessions in one profile, same provider and model:

Session	State	Log JSONL bytes	delivery-accepted watermarks
f7d378ca…	fails 413 (turn 230)	95,881,363 (~91 MiB)	0
93b06c99…	fails 413 (turn 231)	95,912,073 (~91 MiB)	0
4b44862f…	fails (see note)	135,112,977 (~129 MiB)	0
01155af0… (new)	works	1,720,673 (~1.7 MiB)	55

Two further observations:

1. The first attempt of the failing turn reports TRANSPORT, and only the retry gets a
   clean 413 — consistent with an oversized body dying mid-upload.
2. providerError() in packages/llm/llm-deepseek/src/protocols/messages/transport.ts
   tests isContextWindowExceededError(detail) before status === 413. The result was
   INVALID_REQUEST, not CONTEXT_WINDOW_EXCEEDED, so the response carried no
   context-overflow wording — this is a body-size rejection, not a context overflow.

Root cause

@deepseek-ai/dsh-session-log-deepseek contributes the dsh_session_log request-body
field. Its contract is incremental: it sends only the log suffix after the greatest
accepted watermark.

packages/session/session-log-deepseek/src/index.ts computes the pending suffix and
sets throughSeq to the log tail, with no bound on the suffix size:

const throughSeq = snapshot.at(-1)?.seq          // log tail
const suffix = session.snapshotEvents(SessionLogOffset(afterSeq + 1))
const value = { …, throughSeq: Number(throughSeq), events: suffix.map(wireEvent) }

For a session that has never been uploaded there is no watermark, so afterSeq === -1
and the first request carries the complete log — 91 MiB here, embedded as a JSON
member of the request body.

This is a dead end rather than a transient failure: the provider rejects the body, no
2xx follows, accept() never runs, no watermark is recorded, and the next attempt
rebuilds the same oversized body. A session whose log grows past the provider's request
limit is permanently unresumable.

README.md documented exactly this gap under Known Limitations:

No independent request-size cap — complete delivery is fail-closed; provider
rejection leaves the cursor unchanged instead of truncating the log.

Expected

A large pending suffix should drain across successive requests instead of being sent as
one body the provider rejects. dsh_session_log is documented as adding zero
model-input tokens (README.md → Model Experience → Token effect), so bounding it
costs no model context.

Suggested fix

Bound the contributed batch to an oldest-first prefix and make throughSeq name the
last sequence the request actually carries:

export const DEFAULT_MAX_BATCH_BYTES = 4 * 1024 * 1024

// Config gains: maxBatchBytes?: number  (schema default DEFAULT_MAX_BATCH_BYTES)

function takeBatch(events, maxBytes) {
  const batch = []
  let bytes = 0
  for (const event of events) {
    const size = Buffer.byteLength(JSON.stringify(event), 'utf8')
    if (batch.length > 0 && bytes + size > maxBytes) break
    batch.push(event)
    bytes += size
  }
  return batch
}

const events = takeBatch(suffix.map(wireEvent), config.maxBatchBytes ?? DEFAULT_MAX_BATCH_BYTES)
const throughSeq = events.at(-1)?.seq ?? Number(tailSeq)   // last carried, not the tail

Properties:

• Lossless. The batch is a prefix; the watermark advances to its last sequence and
  the next request resumes at the following one. Nothing is skipped or reordered.
• Convergent. One accepted request drains one batch, so a 91 MiB backlog drains in
  ~23 requests instead of failing forever.
• No stall on a single oversized record. The first event is always admitted even if
  it alone exceeds the ceiling; withholding it would freeze the watermark below it.
• Zero model-visible change. The field stays outside messages, the system prompt,
  and tool schemas.

Full diff: 11 files, +162/−20 — <https://github.com/bozhang1214/deepseek-harness/commit/bff8e9a7cf>

Verification

• 2 new regression tests (bounded drain without gaps/duplicates; single oversized event
  still advancing the watermark); package suite 45/45 passing, oxlint clean,
  verify-config-catalog and verify-translation-pairing (919 pairs) clean.
• Committed as bff8e9a7cf; passes the repository's own lefthook gates — pre-commit
  (translation pairing, staged lint, whitespace, vendor manifest guard) and pre-push
  (typecheck).
• Controlled A/B on a real affected session (93b06c99, 91 MB log), counting structured
  413 records only (assistant/attempt / turn/end with failure.status == 413); the
  string "413" in prose gives false positives:
  • maxBatchBytes: 1073741824 (≈ pre-fix behaviour) → 413 reproduces: yes
    (3 real 413s at 20:58:47, 21:00:01, 21:00:58, and the watermark did not advance at all)
  • default 4194304 → the same session recovers: yes
    (watermarks resume at 21:03:02 with 3.2–3.8 MiB batches, and no further 413s)
  • the contributed batch is now 3.6–3.9 MiB per request, versus a single ~91 MB body
  • independently corroborated: the profile patch file's mtime is 21:02:53 — between the
    last 413 (21:00:58) and the recovery (21:03:02)

Impact

Any long-lived session whose log exceeds the provider's request-body limit becomes
permanently unusable on upgrade — and because it fails silently on first upload, users
see only a generic 413 and are likely to misdiagnose it as a context-window problem
(as in #2770) or a compaction problem (as in #2107).

Related, but not duplicate

• 长对话卡住，无法进行下一轮对话：本轮运行失败，触发api 413报错。 #2770 — same 413 symptom, different root cause: provider context overflow on Qwen.
  Here the field is not model input, and a new session on the same model works.
• Web profile ships compaction-basic/command-compact/tool-result-pruner disabled - web sessions never auto-compact, long sessions hit context-exceeded #2107 — web profile auto-compaction disabled. Independent (model-visible context).

Note

While upgrading I also saw UNSUPPORTED_CONTENT ("DeepSeek Messages cannot represent
user/tool-result content reasoning") on a pre-existing session after the protocol: messages default landed — likely a separate migration issue. Happy to file that
separately if useful.

[echoVic]

Corroborating this on a different machine and a later build — same watermark mechanism, but a different gateway failure mode.

dsh 0.1.6-alpha.2 (npm), profile web, deepseek-flash, protocol messages, macOS arm64, Node v22.22.2. Three long-lived sessions deadlocked from the moment of the upgrade restart (09:00:09), while every session created afterwards stayed healthy:

session	log (zstd)	raw JSONL	events	delivery-accepted	symptom
83947afb…	87 MB	452 MB	285,093	0	TRANSPORT ×5, occasionally 550
accc213d…	39 MB	—	128,976	0	TRANSPORT ×5
c115348f…	26 MB	—	118,654	0	TRANSPORT ×5
post-upgrade sessions	≤ 9.6 MB	—	—	17 / 71	healthy

Instead of your 413, the gateway here drops the connection with no HTTP response at all ({"message":"DeepSeek Messages transport failed","code":"TRANSPORT"}), and roughly once per retry storm answers DeepSeek Messages request failed (550). Each attempt runs a very consistent 36–38 s before failing, which reads as an interrupted upload of a few hundred MB. Same deadlock either way: acceptance is only appended after 2xx, so the cursor never moves and every retry resends the whole log.

Two small additions to your write-up:

1. The wire protocol is not the trigger. The new messages default is a red herring here: from the same machine, same key, same minute, /v1/messages and /anthropic/v1/messages both return 200 for a 3.5 MB streaming body, and an over-limit body returns a clean 400 from both. Only requests that carry the whole session log fail. The plugin being default-on over pre-existing sessions is sufficient by itself — those logs have no acceptance events at all, so acceptedThrough() starts at -1.
2. The way out does not need a restart. With patchReload: live (the shipped web profile), adding this to ~/.dsh/profiles/web/cordis.patch.yml takes effect immediately:

- id: session-log-deepseek
  config:
    enabled: false

Both deadlocked sessions produced a normal assistant message again ~40 s after the file was written, with dsh web left running.

More detail, including the ruled-out endpoints and the per-attempt timing, is in #6847.

[webto]

Corroborating on 0.1.7-rc.1 (source checkout, profile web, deepseek-flash, Messages wire protocol, macOS).

A long-lived session in my local profile had 0 session-log-deepseek/delivery-accepted events in either session format generation, so acceptedThrough() returned -1 and every request attached the whole log:

metric	value
events	17,279
dsh_session_log wire bytes	~56.4 MB
assistant/message (reasoning + embedded stream)	39.2 MB
tool/ptc-dispatch + tool/ptc-dispatch-start	10.1 MB
tool/result	2.7 MB

Four notes that may help:

1. Timing fits an oversized upload, not a model rejection. Each failing attempt took a consistent ~9–12 s from request/header to the bare 413, while the session's model-visible surface was only ~1.3 MB (661 surface nodes, 10 images already referenced by Files ids) — the model input is not involved.
2. PTC (run_code) sessions inflate the log. tool/ptc-dispatch* records are ~18% of this log on top of reasoning text in assistant/message, so the first-upload cliff arrives much earlier than the model-visible context suggests.
3. Pre-existing sessions are the trigger, not the wire protocol. This session predates the acceptance events entirely, while sessions created after the same restart accumulate watermarks normally; only pre-existing logs fail.
4. A session-format generation change is a second path to the same deadlock. acceptedThrough() ignores watermarks whose sessionFormatVersion differs from the current generation, so a migrated session also restarts at afterSeq: -1 and re-uploads everything once — worth covering in the same fix.

Workaround confirmed here as well: enabled: false for session-log-deepseek in the profile patch file takes effect without restarting dsh web, and the next turn succeeds.

Adding this as corroboration on 0.1.7-rc.1 rather than opening a duplicate thread.