agent-trace

Debug AI agent sessions by analyzing conversation traces. Find failure patterns, loops, errors, and bottlenecks in your agent workflows.

The Problem

AI agents fail 5-15% of the time with inherent non-determinism. They rabbit-hole, get stuck in loops, and produce errors that are hard to diagnose. When your agent breaks in production, you need to understand why.

The Solution

agent-trace parses agent conversation logs and provides:

• Issue Detection: Tool call loops, error patterns, long reasoning chains, rabbit-holing
• Metrics: Message counts, tool usage distribution, token estimates
• Visualization: Terminal reports and HTML dashboards
• Trace Diffing: Compare two sessions to find divergence points

Installation

npm install -g agent-trace

Or run directly:

npx agent-trace analyze conversation.json

Usage

Analyze a Trace

# Basic analysis
agent-trace analyze session.json

# Verbose output with message timeline
agent-trace analyze session.json --verbose

# Generate HTML report
agent-trace analyze session.json --html

# Output raw JSON
agent-trace analyze session.json --json

Quick Summary

agent-trace summary session.json

Output:

  Quick Summary

  Messages: 47
  Tool Calls: 23
  Errors: 2
  Est. Tokens: ~12,450

  Issues Found: 3
    - TOOL_LOOP: Tool "exec" called 5 times consecutively
    - ERROR: tool_error at message 15
    - RABBIT_HOLE: 28 messages without user input

Compare Two Traces

agent-trace diff session1.json session2.json

Find where two agent runs diverged - useful for debugging non-determinism.

Supported Formats

• OpenAI Chat Format: Messages with role, content, tool_calls
• Anthropic Messages API: Messages with content blocks (text, tool_use, tool_result)
• Generic Event Logs: JSONL with type, event, or similar fields

Format is auto-detected, or specify with --format openai|anthropic|generic.

Example Output

  ═══════════════════════════════════════════
                TRACE ANALYSIS
  ═══════════════════════════════════════════

  Metrics
  ───────────────────────────────────────────
  Messages:        47
  Tool Calls:      23
  Unique Tools:    5
  Errors:          2
  Est. Tokens:     ~12,450

  Tool Usage
  ───────────────────────────────────────────
  exec                       12 ████████████
  read_file                   6 ██████
  write_file                  3 ███
  web_search                  1 █
  browser                     1 █

  Issues Detected
  ───────────────────────────────────────────
  ✖ ERROR
    tool_error at message 15
  ⚠ TOOL_LOOP
    Tool "exec" called 5 times consecutively (messages 20-24)
  ⚠ RABBIT_HOLE
    28 messages without user input (messages 19-46)

  ───────────────────────────────────────────
  Status: ISSUES FOUND

Issue Types

Type	Severity	Description
ERROR	error	Tool returned an error or error detected in output
TOOL_LOOP	warning	Same tool called 3+ times consecutively
RABBIT_HOLE	warning	Long autonomous stretch without user input
LONG_CHAIN	info	Many assistant messages without tool use

Programmatic Usage

import { parseTrace } from 'agent-trace/lib/parser.js';
import { analyzeTrace } from 'agent-trace/lib/analyzer.js';

const trace = parseTrace(conversationData, 'auto');
const analysis = analyzeTrace(trace);

console.log(analysis.issues);
console.log(analysis.metrics);
console.log(analysis.toolStats);

Exporting Traces

From OpenAI

# Save the messages array from your chat completion
import json
with open('trace.json', 'w') as f:
    json.dump(messages, f)

From Anthropic

# Save the messages from your Messages API call
import json
with open('trace.json', 'w') as f:
    json.dump(conversation_messages, f)

From LangChain / LangGraph

Export the message history or use the callback system to capture the trace.

License

MIT