v0.3.0

This release is about meeting traces where they already live. If you are tracing your agent with OpenTelemetry, Langfuse, or LangSmith, you no longer need to re-instrument anything to diff it.

whatbroke import converts existing trace exports into diffable JSONL. It reads OTLP JSON span exports that follow the GenAI semantic conventions, which covers the collector's file exporter, Vercel AI SDK telemetry, Claude Code, and OpenLLMetry. It also reads Langfuse observation exports and LangSmith run dumps. Format is auto-detected from the file, and the import tells you which fields the source did not carry so you know which findings cannot show up.

Langfuse imports keep cost data, so cost regressions work there out of the box. Latency is recomputed from timestamps because the exported latency field changed units between Langfuse versions.

--fail-on changed is now accepted alongside breaking and never, so argument drift can fail a CI build, not just breaking findings.

87 tests. Published on npm as whatbroke-cli.

v0.2.0

This release rounds out the core loop: record a trace, change something, diff.

The diff engine aligns runs by id and tool calls within each run, then reports dropped and reordered calls, argument drift, and cost and latency regressions.

The Recorder SDK wraps existing OpenAI and Anthropic clients, so llm calls and tool calls land in the trace without any manual logging.

whatbroke record: a local proxy that captures traces with zero code changes. Point OPENAI_BASE_URL or ANTHROPIC_BASE_URL at it, run your agent, done. Streaming responses pass through untouched and still get parsed.

Multi-sample runs: name your runs refund-flow#1, refund-flow#2 and whatbroke compares every before sample against every after sample. Findings carry an occurrence rate like 6/9, and anything that already flaps between baseline samples is demoted to flaky info instead of being blamed on your change.

Published on npm as whatbroke-cli.

v0.1.0

First release.

whatbroke diffs two recorded runs of an AI agent and reports what actually changed: dropped tool calls, drifted arguments, new errors, output flips, latency and cost regressions. Deterministic, offline, no API keys.

• whatbroke diff before.jsonl after.jsonl with terminal, --json and --md reports
• CI-friendly exit codes via --fail-on breaking|warning|never
• Recorder SDK that wraps OpenAI and Anthropic clients in one line
• Plain JSONL trace format you can write from any language
• Bundled example: a support agent before and after a cheap model swap, including the part where it quietly stops cancelling subscriptions