Skip to content

feat(transcript): add canonical agent exchange transcript (F106)#370

Merged
pocky merged 1 commit into
mainfrom
feature/F106-canonical-agent-exchange-transcript-json
Jun 8, 2026
Merged

feat(transcript): add canonical agent exchange transcript (F106)#370
pocky merged 1 commit into
mainfrom
feature/F106-canonical-agent-exchange-transcript-json

Conversation

@pocky

@pocky pocky commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Add F106: a canonical agent exchange transcript system that records every step of workflow execution — user messages, agent responses, tool calls and results — as an append-only JSONL file
  • Introduce a provider-agnostic ContentBlock normalisation layer that translates raw Claude, Gemini, Codex, Copilot, and OpenAI-compatible output into a unified schema before writing
  • Wire the transcript recorder through the full execution stack (tool proxy, subworkflow executor, conversation manager) so that complex flows including nested call-workflow steps and loops produce a single, replayable transcript per run
  • Ship a FanoutRecorder that multiplexes writes to multiple backends (file + future sinks) without coupling execution logic to any specific output destination

Changes

Domain — transcript schema

  • internal/domain/transcript/content.go: ContentBlock union type covering text, tool-use, tool-result, thinking, and error variants
  • internal/domain/transcript/event.go: Event envelope (role, content blocks, metadata, run ID) that forms one line in the JSONL file
  • internal/domain/transcript/payload.go: payload helper types shared by event constructors
  • internal/domain/transcript/doc.go: package documentation and architecture rationale

Domain — ports

  • internal/domain/ports/recorder.go: Recorder port interface (Emit / Close); consumed by application layer
  • internal/domain/ports/agent_output_normalizer.go: AgentOutputNormalizer port for provider-to-ContentBlock translation

Application — execution wiring

  • internal/application/execution_service.go: integrate recorder into the main agent step loop; emit user, assistant, and error events
  • internal/application/execution_service_transcript.go: emitTranscriptAgentResponse helper; normalises agent output and writes an assistant event
  • internal/application/execution_setup.go: RecorderFactory wiring; inject recorder into ExecutionService via SetRecorder
  • internal/application/execution_tool_proxy.go: intercept tool calls and results through the proxy; emit tool-use and tool-result events
  • internal/application/subworkflow_executor.go: propagate recorder into nested call_workflow executions
  • internal/application/conversation_manager.go: expose run ID accessor needed by transcript event construction
  • internal/application/tools/router.go: pass recorder reference through the tool router for tool-level recording

Infrastructure — transcript I/O

  • internal/infrastructure/transcript/recorder.go: JSONLRecorder — writes events as newline-delimited JSON to a file per run ID
  • internal/infrastructure/transcript/jsonl_writer.go: low-level atomic append writer with fsync guarantees
  • internal/infrastructure/transcript/fanout.go: FanoutRecorder — fans out Emit calls to N child recorders; closes all on Close
  • internal/infrastructure/transcript/reader.go: Reader for replaying a transcript file event-by-event

Infrastructure — provider normalisers

  • internal/infrastructure/agents/content_block_normalizer.go: ContentBlockNormalizer dispatches to per-provider functions
  • internal/infrastructure/agents/claude_to_content_blocks.go: Claude JSONL → ContentBlock
  • internal/infrastructure/agents/codex_to_content_blocks.go: Codex JSONL → ContentBlock
  • internal/infrastructure/agents/copilot_to_content_blocks.go: Copilot JSONL → ContentBlock
  • internal/infrastructure/agents/gemini_to_content_blocks.go: Gemini JSONL → ContentBlock
  • internal/infrastructure/agents/openai_compatible_to_content_blocks.go: OpenAI-compatible JSONL → ContentBlock
  • internal/infrastructure/agents/base_cli_provider.go: expose output lines to normaliser
  • internal/infrastructure/agents/testdata/transcript/*/input.jsonl: per-provider raw fixture files
  • internal/infrastructure/agents/testdata/transcript/golden/*.json: golden expected ContentBlock outputs

CLI wiring

  • internal/interfaces/cli/run.go: pass transcript recorder factory into ExecutionSetup.Build
  • internal/interfaces/cli/wiring_transcript.go: BuildTranscriptRecorder — resolves storage path, creates JSONLRecorder + optional fanout

Tests

  • internal/application/execution_service_transcript_test.go: unit tests for assistant event emission
  • internal/application/execution_service_transcript_assistant_test.go: assistant content-block normalisation coverage
  • internal/application/execution_service_transcript_callworkflow_test.go: nested call-workflow transcript propagation
  • internal/application/execution_service_transcript_loop_test.go: loop step transcript continuity
  • internal/application/execution_service_runid_test.go: run ID generation and uniqueness
  • internal/application/execution_setup_recorder_factory_test.go: recorder factory injection
  • internal/application/execution_setup_test.go: updated setup wiring tests
  • internal/application/execution_tool_proxy_recorder_test.go: tool proxy recording assertions
  • internal/application/tools/router_transcript_test.go: router-level recording tests
  • internal/domain/ports/recorder_contract_test.go: port contract tests (Emit idempotency, Close behaviour)
  • internal/domain/transcript/content_test.go: ContentBlock construction and serialisation
  • internal/domain/transcript/event_test.go: Event serialisation round-trips
  • internal/domain/transcript/doc_test.go: package example tests
  • internal/domain/transcript/architecture_test.go: dependency rule enforcement
  • internal/infrastructure/agents/*_test.go: per-provider normaliser unit tests
  • internal/infrastructure/agents/golden_transcript_test.go: golden-file regression suite
  • internal/infrastructure/transcript/fanout_test.go: fanout multiplexing and error propagation
  • internal/infrastructure/transcript/jsonl_writer_test.go: writer append / close semantics
  • internal/infrastructure/transcript/jsonl_writer_atomicity_test.go: crash-safety and atomicity tests
  • internal/infrastructure/transcript/reader_test.go: reader replay correctness
  • internal/infrastructure/transcript/recorder_test.go: full recorder lifecycle
  • internal/interfaces/cli/wiring_transcript_test.go: CLI wiring integration
  • tests/integration/features/transcript_test.go: end-to-end transcript creation via awf run

Configuration

  • .go-arch-lint.yml: register internal/domain/transcript and internal/infrastructure/transcript with their allowed dependency sets
  • .golangci.yml: add transcript packages to linter scope

Documentation

  • docs/reference/transcript-schema.md: full JSONL schema reference with field descriptions and examples
  • docs/user-guide/transcript.md: user-facing guide — enable, locate, and interpret transcripts
  • docs/user-guide/audit-trail.md: updated to reference transcript as the audit mechanism
  • docs/README.md: link to new transcript docs
  • README.md: mention transcript feature in feature list

ZPM knowledge base

  • .zpm/kb/feedback/journal.wal: feedback rules learned during F106 implementation
  • .zpm/kb/pr_feature_f106_canonical_agent_exchange_transcript_j/journal.wal: PR-scoped tracking facts
  • .zpm/kb/pr_feature_f106_canonical_agent_exchange_transcript_j/knowledge.pl: compiled knowledge snapshot

Test plan

  • make build completes without errors; awf run produces a .jsonl file under storage/transcripts/<run-id>.jsonl
  • make test-unit passes with zero failures; verify transcript-related packages in output
  • make test-integration passes tests/integration/features/transcript_test.go; confirm file contains valid JSON lines with correct roles
  • make lint reports zero violations for new packages

Closes #369


Generated with awf commit workflow

- `.go-arch-lint.yml`: Register transcript domain and infrastructure packages with dependency rules
- `.golangci.yml`: Add linter configuration for new transcript packages
- `docs/reference/transcript-schema.md`: Add full transcript JSON schema reference
- `docs/user-guide/audit-trail.md`: Document transcript as audit trail mechanism
- `docs/user-guide/transcript.md`: Add user guide for transcript feature
- `internal/domain/ports/recorder.go`: Add Recorder port interface for append-only event recording
- `internal/domain/ports/recorder_contract_test.go`: Add contract tests for Recorder port
- `internal/domain/ports/agent_output_normalizer.go`: Add AgentOutputNormalizer port interface
- `internal/domain/transcript/event.go`: Add canonical transcript event types and schema
- `internal/domain/transcript/content.go`: Add content block types for provider-agnostic output
- `internal/domain/transcript/payload.go`: Add payload types for transcript events
- `internal/domain/transcript/doc.go`: Add package documentation with architecture overview
- `internal/domain/transcript/architecture_test.go`: Add architecture constraint tests
- `internal/domain/transcript/event_test.go`: Add event type unit tests
- `internal/domain/transcript/content_test.go`: Add content block unit tests
- `internal/domain/transcript/doc_test.go`: Add doc example tests
- `internal/domain/workflow/agent_config.go`: Add RunID field for transcript correlation
- `internal/domain/workflow/context.go`: Expose RunID accessor for execution context
- `internal/domain/workflow/conversation.go`: Wire RunID into conversation context
- `internal/infrastructure/agents/claude_to_content_blocks.go`: Normalize Claude output to canonical content blocks
- `internal/infrastructure/agents/codex_to_content_blocks.go`: Normalize Codex output to canonical content blocks
- `internal/infrastructure/agents/copilot_to_content_blocks.go`: Normalize Copilot output to canonical content blocks
- `internal/infrastructure/agents/gemini_to_content_blocks.go`: Normalize Gemini output to canonical content blocks
- `internal/infrastructure/agents/openai_compatible_to_content_blocks.go`: Normalize OpenAI-compatible output to canonical content blocks
- `internal/infrastructure/agents/content_block_normalizer.go`: Add ContentBlockNormalizer aggregating all provider converters
- `internal/infrastructure/agents/base_cli_provider.go`: Wire AgentOutputNormalizer into base provider
- `internal/infrastructure/agents/testdata/transcript/`: Add golden test fixtures for all five providers
- `internal/infrastructure/agents/*_test.go`: Add unit and golden tests for all provider normalizers
- `internal/infrastructure/transcript/recorder.go`: Add JSONL-backed Recorder implementation
- `internal/infrastructure/transcript/jsonl_writer.go`: Add atomic append-only JSONL writer
- `internal/infrastructure/transcript/fanout.go`: Add FanoutRecorder for multi-sink event dispatch
- `internal/infrastructure/transcript/reader.go`: Add transcript JSONL reader for replay
- `internal/infrastructure/transcript/doc.go`: Add package documentation
- `internal/infrastructure/transcript/*_test.go`: Add unit tests including atomicity and fanout tests
- `internal/application/execution_service.go`: Integrate Recorder into execution lifecycle
- `internal/application/execution_service_transcript.go`: Add transcript emission helpers for all event types
- `internal/application/execution_setup.go`: Wire RecorderFactory into ExecutionSetup.Build()
- `internal/application/execution_tool_proxy.go`: Emit tool call and result events to transcript
- `internal/application/subworkflow_executor.go`: Emit subworkflow events to transcript
- `internal/application/tools/router.go`: Forward transcript recorder through tool router
- `internal/application/conversation_manager.go`: Propagate recorder into conversation manager
- `internal/application/execution_service_runid_test.go`: Add RunID propagation tests
- `internal/application/execution_service_transcript_test.go`: Add transcript emission unit tests
- `internal/application/execution_service_transcript_assistant_test.go`: Add assistant response emission tests
- `internal/application/execution_service_transcript_callworkflow_test.go`: Add subworkflow transcript tests
- `internal/application/execution_service_transcript_loop_test.go`: Add loop transcript tests
- `internal/application/execution_setup_recorder_factory_test.go`: Add recorder factory wiring tests
- `internal/application/execution_tool_proxy_recorder_test.go`: Add tool proxy recorder tests
- `internal/application/tools/router_transcript_test.go`: Add router transcript forwarding tests
- `internal/interfaces/cli/wiring_transcript.go`: Add CLI wiring for transcript recorder factory
- `internal/interfaces/cli/run.go`: Wire transcript recorder into run command
- `tests/integration/features/transcript_test.go`: Add end-to-end transcript integration tests

Closes #369
@pocky pocky marked this pull request as ready for review June 8, 2026 12:21
@pocky pocky merged commit bb7b0a2 into main Jun 8, 2026
5 checks passed
@pocky pocky deleted the feature/F106-canonical-agent-exchange-transcript-json branch June 8, 2026 12:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

F106: Canonical Agent Exchange Transcript (JSONL)

1 participant