feat(metrics): expose reasoning_tokens in LLM metrics and traces

[s-hamdananwar]

Carries reasoning_tokens from OpenAI-compatible usage payloads through to LLMMetrics, LLMModelUsage, the log_metrics line, and a new gen_ai.usage.reasoning_tokens span attribute. Reasoning is a subset of completion_tokens, never additive.

The inference gateway already returns completion_tokens_details.reasoning_tokens, but inference/llm.py never read it, so the value was dropped before reaching metrics or traces.

[devin-ai-integration]

Devin Review found 1 potential issue.

[devin-ai-integration]

🟡 Remote sessions lose reasoning usage

output_reasoning_tokens has no matching field in encode_session_usage. Remote usage events and queries silently omit every reasoning-token count.

Prompt for agents

The new LLMModelUsage.output_reasoning_tokens value is retained in local session usage and reports, but livekit-protocol 1.1.21's LLMModelUsage protobuf has no output_reasoning_tokens field. encode_session_usage in livekit-agents/livekit/agents/_proto.py copies only fields declared by that protobuf, so RemoteSession session_usage_updated events and get_session_usage responses drop the value. Add the corresponding field to the agent-session protocol schema and update the protocol dependency/generated bindings, then add coverage that starts with a nonzero Python output_reasoning_tokens value and verifies it survives encode_session_usage.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[shawnfeldman]

Clean additive change — every new field defaults to 0, so nothing that doesn't report reasoning shifts.

Two things I checked rather than assumed:

The test really does exercise the changed code. tests/test_inference_llm_usage.py imports livekit.plugins.openai, while the wiring lands in livekit-agents/.../inference/llm.py — which reads like the test covering a different path. It isn't: the plugin's LLMStream (llm.py:1041) subclasses inference.llm.LLMStream and overrides only __init__, so it inherits the changed _run. Mutation-confirmed — dropping reasoning_tokens=reasoning_tokens or 0, from the CompletionUsage construction fails test_reasoning_tokens_are_reported_from_completion_tokens_details with assert 0 == 32, baseline 3 passed. reasoning_tokens appears in exactly the nine files this PR touches, so that ctor is the only producer.

Field insertion order is safe. reasoning_tokens: int = 0 goes in ahead of total_tokens: int (no default) in both LLMMetrics and CompletionUsage. That would be a TypeError at import in a dataclass; both are Pydantic BaseModels, and cache_creation_tokens: int = 0 already sits in the same position, so the shape is pre-existing.

The subset-not-additive invariant is stated in all three docstrings and pinned by test_collector_aggregates_reasoning_tokens asserting output_tokens == 150 alongside output_reasoning_tokens == 72. Adding the new span attribute to the test_trace_types_pii.py allowlist is right — a token count carries no PII. Gating the span set_attribute on truthiness keeps non-reasoning spans unchanged.

25/25 checks green, including unit-tests.

Two notes, neither blocking and both arguably out of scope: RealtimeModelMetrics in ModelUsageCollector.collect has no equivalent branch, so realtime usage will report output_reasoning_tokens == 0 even for a reasoning model; and plugins that build CompletionUsage outside the inference.llm.LLMStream base class won't populate it.