v0.1.9 - Complete Metrics Implementation
Summary
End‑to‑end metrics support for Traccia: standardized LLM + agent metrics across core SDK, integrations, and docs, with a central recorder, pricing awareness, and tests.
Highlights
Core metrics system
- Introduced a unified metrics layer (metrics.metrics, metrics.recorder) to emit:
- Token usage
- Operation cost (USD)
- Operation duration / latency
- Added a global metrics recorder accessor so integrations can record metrics without tight coupling.
SDK & instrumentation
- Updated core instrumentation (auto.py, config.py, instrumentation/openai.py, instrumentation/anthropic.py, instrumentation/requests.py) to:
- Attach pricing‑aware cost metrics to each LLM call.
- Record token usage and duration consistently under gen_ai.client.*.
- Exposed metrics configuration and behavior at the top level (init.py) for easier imports and discoverability.
Framework integrations
- LangChain: Extended callback handler to emit standardized metrics (tokens, cost, duration) for LangChain LLM calls.
- CrewAI: Instrumented CrewAI integration to record LLM metrics via the central recorder.
- OpenAI Agents SDK: Enhanced processor to:
- Track span start times.
- Record generation metrics (tokens, cost, duration) on span end using the new metrics layer.
CLI & configuration
- Updated cli.py and config.py to:
- Include metrics options in configuration handling.
- Ensure traccia config init produces a config that is compatible with the new metrics system.
Testing & docs
- Added tests/test_metrics.py with coverage for:
- Metrics recorder behavior.
- Emission of token, cost, and duration metrics under expected names/attributes.
- Expanded README.md with:
- An overview of built‑in metrics (LLM + agent).
- How metrics relate to pricing and token usage.
- Guidance on enabling metrics in typical setups.