Skip to content

D-8: ClaudeStreamJSONParser — JSONL → AgentStreamEvent #258

@kirich1409

Description

@kirich1409

Description

Implement the parser that converts Claude Code CLI stream-json (NDJSON) into the neutral AgentStreamEvent type defined in D-7. This is the schema-translation layer — all Claude-specific knowledge lives here and nowhere else in the codebase.

Spec: docs/architecture/dialogue-events.md (full event catalog), Epic #250 §7.

Scope

File layout

MacApp/Packages/AgentChat/Sources/AgentChat/Parser/

  • ClaudeStreamJSONParser.swift — main actor
  • ClaudeStreamJSONDTO.swift — wire-level Codable DTOs (one file per group: SystemMessage, AssistantMessage, UserMessage, ResultMessage, StreamEvent)
  • ParserBuffer.swift — line buffering until \n, 10 MiB safeguard
  • ToolKindInference.swift — map Claude tool names (Read/Edit/Bash/mcp__*/...) to ToolKind

Parser contract

```swift
public actor ClaudeStreamJSONParser {
public init(sessionID: SessionID)

/// Feed bytes from TerminalSession.rawStdout. Emits events on the returned stream.
public func events() -> AsyncStream<AgentStreamEvent>

/// Feed raw bytes (may be partial; parser buffers until \\n).
public func ingest(_ data: Data)

/// Signal source closed (process exit). Flushes + emits terminated event if no result seen.
public func close()

}
```

Parsing rules

  • NDJSON: buffer until \n, parse each line with JSONDecoder.
  • Line limit: 10 MiB — drop with warning, continue from next \n.
  • Tolerant decoding: unknown fields ignored via default Codable; unknown event types → emit AgentStreamEvent.unknownEvent(raw: String).
  • Top-level dispatch on type: system / assistant / user / result / stream_event.
    • system → switch subtype → emit matching event.
    • assistant → parse message.content[], emit messageCompleted with usage/stopReason.
    • user → detect tool_result vs echo, emit toolCallCompleted / userMessage.
    • result → emit sessionEnd(SessionResult), mark parser state as terminal.
    • stream_event → switch inner event.type → emit textDelta / thinkingStarted / toolCallStarted / etc.
  • Dedup in parser: track composite keys (see events spec §3). LRU buffer of 2000 keys.
  • Out-of-order content_block_* interleaving (rare) — defensive support: track buffers per index.

Dispatch special cases

  • tool_use.nameToolKind mapping table (source: events spec §4).
  • mcp__<server>__<tool> names → ToolKind.other with metadata {server, tool}.
  • EnterPlanMode / ExitPlanMode as system events, not tool cards.

Acceptance Criteria

  • Parser implemented per contract; all event types from events spec §2 covered.
  • Fixture tests (Tests/AgentChatTests/Parser/Fixtures/) — minimum 20 real JSONL samples covering: basic text response, interleaved tool_use + tool_result, thinking, api_retry, compact_boundary, rate_limit, subagent (parent_tool_use_id), MCP tool, failed tool, unknown event type.
  • Partial-line test — feed a multi-event stream byte-by-byte (simulated slow stream) and verify correct parsing.
  • Dedup test — feed duplicate events, verify parser emits each unique event exactly once.
  • Overflow test — 11 MiB single line drops with warning, next events parse normally.
  • Tolerant decoding — add an unknown field to an assistant event; parser decodes known fields, ignores unknown, no exception.
  • Benchmark: parse 10k events in < 1 second on M-series Mac (rough sanity; not a hard SLA).
  • Swift 6 strict concurrency clean.

Relationships

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions