[Bug] Retry attempts and compaction requests are undercounted in session usage

[stoneface10]

Summary / 摘要

The durable token/session projections can under-report provider requests when retries and context compaction are involved:

1. A failed attempt's usage can be replaced by a successful retry's usage because retries run inside the same {turn, step}.
2. A scheduled retry can be counted even when cancellation during backoff prevents the repeated request from starting.
3. compaction/summary provider usage is durable but is omitted from whole-session token totals.

These are provider-agnostic accounting issues. They became visible while correlating DSH statistics with subscription-backed model usage.

DSH 当前在 Retry 与 Context Compaction 场景下可能低估完整 provider 请求消耗：失败 attempt 的 usage 被成功 retry 覆盖、退避期间取消的 retry 被当成已执行、Compaction 摘要请求未进入会话总计。

Minimal event sequences

1. Failed attempt followed by successful retry

step/start { turn: 1, step: 1 }
assistant/chunk usage A
llm/retry
llm/retry-started
assistant/chunk usage B
assistant/message usage B
step/end

Retries execute inside the same step. If usage identity is only step:<turn>:<step>, B replaces A. Expected whole-session billing is A + B; last-sample replacement should apply only within one request attempt.

2. Cancellation during backoff

llm/retry          # retry scheduled
... cancellation during delay ...
step/end
turn/end aborted

No llm/retry-started occurs and no repeated provider request starts. Counting llm/retry reports one retry although only the original attempt ran.

3. Compaction summary usage

compaction/summary {
  provider,
  model,
  usage,
  ...
}

This is a separate provider request and should contribute to whole-session usage. It should not replace the main model's context-pressure sample because it belongs to the summarizer request.

Proposed accounting rules

• Continue last-sample replacement for usage chunks/final messages within one attempt.
• On llm/retry-started, close/reset replacement adjacency so the next attempt accumulates independently.
• Count actual retries from llm/retry-started, not scheduled backoffs from llm/retry.
• Add compaction/summary.usage to tokenUsage, keyed by compactionId.
• Keep compaction usage out of contextPressure.
• Mirror the production fold in the client fixture.
• Display actual retry attempts and uncached input in the conversation stats row.

Validation

Focused patch validation:

• token-meter: failed-attempt usage + successful retry regression
• session-stats: scheduled-but-cancelled retry regression
• client fixture: compaction billing and retry-boundary parity
• conversation UI: only started retries are counted
• 4 test files, 96 tests passed
• TypeScript project build passed
• Broader integrated modified-package suite previously passed 260 tests
• Independent review found no remaining blockers

Reference implementation

Focused bug-fix branch and commit:

• https://github.com/stoneface10/deepseek-harness/tree/fix/complete-token-accounting
• ccc842215a

An integrated branch that also contains the separately reported Windows fix is available here:

• https://github.com/stoneface10/deepseek-harness/tree/feat/codex-quota-and-windows-fixes

Because external PRs are currently not accepted, this Discussion provides reproducible event sequences, semantics, tests, and a focused patch. I would appreciate confirmation of the intended accounting semantics and whether maintainers would like to import or reimplement it.

[Electricitysheep]

Retry 和 compaction 请求未计入 session usage——成本统计口径漏了重试和压缩的消耗，会让成本账单偏低。

成本透明化（第 14 章）的前提是统计准确，这类口径问题该修：https://github.com/Electricitysheep/dsh-handbook/blob/main/docs/14-cost.md

[lizhuojunx86]

Two implementations of this exist now, and they differ in one structural way worth deciding before either is imported.

Your rule 2 closes replacement adjacency on llm/retry-started. 63688b0 (in #1886) closes it on the failed finish chunk itself — assistant/chunk with reason kind error | aborted — clearing last only when the finish's own (turn, step) matches the open sample.

I verified 63688b0 against a real corpus rather than against its description: whole log 308,234, child-owned 55,886, bucket for bucket, identical to an independent fold written from the source. Both approaches produce the right number on the retry path. The difference is what they depend on.

The finish chunk is already in the stream usageOf() reads. Keying on llm/retry-started makes token-meter depend on retry semantics owned by another package, and it only fires when the retry subsystem emits those events — an adapter that retries internally, or a failed attempt with no retry after it, never produces one. The finish boundary covers both without knowing that retries exist.

On your rules 2 and 3 being separate concerns, which I think is the most useful thing in this thread: usage settlement and retry counting are two different quantities and should not share a discriminator. A failure finish settles a billed attempt — that is money. llm/retry-started counts a retry that actually began — that is a reliability metric. A backoff cancellation, where llm/retry fires with no llm/retry-started, must not increment the second and has no effect on the first. Folding both onto one event is how you end up with a system where fixing the retry count moves the bill.

Rules 4 and 5 match what #1886 measured: 3 compaction events, 48,895 tokens, none counted, and the largest was cache-read heavy enough to matter.

One boundary that neither implementation exercises yet, now split out as #3126: FinishReasonMap is merge-extensible but AgentLoop only allowlists error | aborted, so a third failure kind still assembles an assistant message. Widening the token-meter predicate alone converts this undercount into a double count.