[BUG] JSONL session transcripts missing final message_stop event, output token counts unreliable

[monk1337]

Preflight Checklist

• I have searched existing issues and this hasn't been reported yet
• This is a single bug report (please file separate reports for different bugs)
• I am using the latest version of Claude Code

What's Wrong?

Session JSONL files (~/.claude/projects/<project>/<session-id>.jsonl) log every content block during streaming but never log
the final message_stop event.

• output_tokens in the usage field is a mid-stream snapshot, not the final count
• stop_reason is always null, no line ever records "end_turn"
• Output tokens are undercounted by ~2x compared to actual content
• Input/cache token counts are accurate (set at request start)

Evidence

Single API call producing thinking + text + 3 tool_use blocks:

Line 41: thinking    output_tokens=8     stop_reason=null
Line 42: text        output_tokens=8     stop_reason=null
Line 43: tool_use    output_tokens=8     stop_reason=null
Line 45: tool_use    output_tokens=8     stop_reason=null
Line 46: tool_use    output_tokens=310   stop_reason=null  ← partial update, still not final

Full session comparison (128 API calls, Opus 4.6, 49 min):

┌───────────────────────────────────────────┬───────────────┐
│                  Method                   │ Output Tokens │
├───────────────────────────────────────────┼───────────────┤
│ JSONL output_tokens summed                │ 23,725        │
├───────────────────────────────────────────┼───────────────┤
│ tiktoken cl100k_base on extracted content │ 45,050        │
├───────────────────────────────────────────┼───────────────┤
│ Undercount                                │ ~1.9x         │
└───────────────────────────────────────────┴───────────────┘

Impact

- Cost tracking tools (ccusage, cccost, custom scripts) that parse JSONL get wrong output counts
- No way to reconcile per-session cost against Anthropic billing dashboard
- Users building usage reports or budgeting around JSONL data are underestimating spend

### What Should Happen?

Each API call should have a final JSONL line with:
- `stop_reason: "end_turn"` (or `"tool_use"`)
- Accurate final `output_tokens` count matching what Anthropic actually bills

This is one additional JSONL line per API call , the final streaming event that currently gets discarded.

### Error Messages/Logs

```shell
{
  "type": "assistant",
  "requestId": "req_011CYLnnca3yt6Axy4nsUrRM",
  "message": {
    "role": "assistant",
    "stop_reason": null,
    "usage": {
      "input_tokens": 1,
      "cache_creation_input_tokens": 510,
      "cache_read_input_tokens": 40716,
      "output_tokens": 310
    }
  }
}

No line in the entire 1043-line JSONL transcript has stop_reason set to anything other than null.

Steps to Reproduce

1. Run any Claude Code session with multiple tool calls
2. After session ends, open the JSONL transcript at ~/.claude/projects/<project>/<session-id>.jsonl
3. Search for stop_reason , every line shows null
4. Sum output_tokens from the last JSONL line per unique requestId
5. Compare against actual output content (extract all thinking/text/tool_use blocks, tokenize with tiktoken)
6. Observe ~2x undercount

Claude Model

Opus

Is this a regression?

No, this never worked

Last Working Version

No response

Claude Code Version

2.1.50

Platform

Anthropic API

Operating System

macOS

Terminal/Shell

Terminal.app (macOS)

Additional Information

Session analyzed: 128 API calls, 49 minutes, ~11.3M total input tokens (mostly cached), heavy MCP tool usage. The undercount is consistent across the entire session, not isolated to specific request types.

[github-actions]

Found 3 possible duplicate issues:

1. [BUG] JSONL logs record partial output_tokens from message_start events instead of final totals #22671
2. [BUG] Output tokens incorrectly recorded in JSONL - only partial streaming values saved #22686
3. JSONL session logs record placeholder output_tokens (always 1-2) instead of real values #25941

This issue will be automatically closed as a duplicate in 3 days.

• If your issue is a duplicate, please close it and 👍 the existing issue instead
• To prevent auto-closure, add a comment or 👎 this comment

🤖 Generated with Claude Code

[monk1337]

Built a tool for this, claudeoo. https://github.com/monk1337/claudeoo/tree/main It wraps claude and intercepts the full SSE stream to capture accurate final token counts (the
built-in tracking logs mid-stream snapshots, which undercounts output tokens by ~2x).

  npm install -g claudeoo
  claudeoo  # drop-in replacement, all args pass through

Gives you per-turn breakdowns, session reports, and a query CLI. Open source, zero dependencies.

[sstklen]

Hey! I ran into a similar pattern in our bug knowledge base and thought this might help.

What's happening: Claude Code's JSONL session transcript writer captures content_block events during SSE streaming but does not log the final message_stop event from the Anthropic API. The output_tokens and stop_reason fields are written from mid-stream message_delta snapshots (which carry partial/cumulative counts), but the definitive final usage object sent with the message_stop SSE event is never persisted. This means the last logged output_tokens value is whatever partial count was attached to the last content_block processed, not the API's authoritative final tally.

What worked for us:

The SSE stream handler that writes to the session JSONL file needs to also listen for the message_stop event (or equivalently the final message_delta with stop_reason: 'end_turn' and complete usage) and write one additional JSONL line containing the finalized output_tokens, stop_reason, and complete usage breakdown. This is a Claude Code internal bug — users cannot patch it directly but can use the community workaround claudeoo (npm install -g claudeoo) which wraps Claude Code and intercepts the full SSE stream to capture accurate final token counts.

Steps:

1. In the SSE stream handler (session transcript writer), add a handler for the message_stop event type
2. When message_stop is received, extract the final usage object which contains the authoritative output_tokens count
3. Write one final JSONL line with type='message_complete', the final output_tokens, stop_reason='end_turn', and full usage breakdown (input_tokens, output_tokens, cache_creation_input_tokens, cache_read_input_tokens)
4. For backward compatibility, keep existing content_block logging but mark those output_tokens as 'partial' or 'streaming_snapshot'
5. Workaround for users: install claudeoo (npm install -g claudeoo) as a drop-in wrapper that captures accurate final token counts from the SSE stream

// Conceptual fix for Claude Code's SSE stream → JSONL writer
// BEFORE (current behavior - only logs content blocks):
stream.on('content_block_start', (block) => {
  appendToJsonl({ type: block.type, output_tokens: message.usage?.output_tokens, stop_reason: null });
});

// AFTER (also capture the final message_stop event):
stream.on('content_block_start', (block) => {
  appendToJsonl({ type: block.type, output_tokens: message.usage?.output_tokens, stop_reason: null, token_status: 'streaming' });
});

stream.on('message_stop', () => {
  // message_stop carries the FINAL usage with authoritative output_tokens
  appendToJsonl({
    type: 'message_complete',
    output_tokens: message.usage.output_tokens,  // final count
    stop_reason: message.stop_reason,             // 'end_turn'
    usage: { ...message.usage },                  // full breakdown
    token_status: 'final'
  });
});

// USER WORKAROUND (until official fix):
// npm install -g claudeoo && claudeoo  # drop-in replacement with accurate token tracking

Hope this helps! Let me know if it doesn't match your case — happy to dig deeper. 🦞

Disclosure: This analysis is from Confucius Debug, an AI-powered community KB for agent bugs. Please verify before applying.

---

_{🦞 Confucius Debug — community knowledge base for AI agent bugs. Free to search via MCP.}

[github-actions]

Closing for now — inactive for too long. Please open a new issue if this is still relevant.

[github-actions]

This issue has been automatically locked since it was closed and has not had any activity for 7 days. If you're experiencing a similar issue, please file a new issue and reference this one if it's relevant.