#1604 Track LLM and embedding cost per individual call

[azanux]

This PR closes the observability part of #1604. The goal is to track cost per individual LLM call.

Today, usage is recorded once per tool-loop as a single aggregate. This breaks down in CONCURRENT mode . The ticket needs LlmMetadata, Usage, interactionId and agentProcess.id available for each call.

The PR adds two new events: LlmInvocationEvent and EmbeddingInvocationEvent. Each fires once per call, with the four required fields. Listeners can aggregate cost as they need.

The implementation reuses an existing extension point. A ToolLoopInspector is created inline at each toolLoopFactory.create(...) site in ToolLoopLlmOperations. It captures llm and llmRequestEvent via closure. DefaultToolLoop fires it through its afterLlmCall hook. The tool-loop core is untouched.

Small side refactor: doTransformIfPossible was inlining start/completed event logic. It now uses the shared publishToolLoopStartEvent and handleToolLoopCompletion helpers, like the three other doTransform* methods. This addresses the "seems one call is missing" review comment of @igordayen

The post-loop recordUsage(...) was removed from handleToolLoopCompletion. Keeping it would double-count on top of the per-call inspector.

EmbeddingOperations emits EmbeddingInvocationEvent after each recorded invocation, same pattern.

---

Behavior change

agentProcess.llmInvocations now contains N entries per loop (one per call) instead of one aggregate. Sums (cost(), usage()) are unchanged. Tests asserting size == 1 on loop scenarios were updated. No production caller depends on the aggregate semantics.

---

Sample listener

class OrganizationCostTracker : AgenticEventListener {
    private val costPerAgent = ConcurrentHashMap<String, DoubleAdder>()

    override fun onProcessEvent(event: AgentProcessEvent) {
        if (event is LlmInvocationEvent) {
            costPerAgent
                .computeIfAbsent(event.agentProcess.agent.name) { DoubleAdder() }
                .add(event.invocation.cost())
        }
    }
}

In CONCURRENT mode the event fires from multiple threads. Stateful listeners must be thread-safe.

---

Not in this PR

Pre-call enforcement #1597 (blocking when a usage limit is hit) is out of scope.but this feature solve it.

Streaming is deferred. Stefan confirmed it isn't needed for v0.5.

Observability module - a full update will be handled in a dedicated PR to account for the changes introduced here and handle in the same time issue #1561 and #1588

---

Test plan

• LlmInvocationEventTest / EmbeddingInvocationEventTest - event shape and fields
• ChatClientLlmTransformerTest - per-call count + matching interactionId
• EmbeddingOperationsTest - emission with/without active agent
• ToolLoopLlmOperationsTest.BillingInspectorTests - per-call emission across the 4 doTransform* paths, listener-cost aggregation, concurrency (16 parallel calls produce 16 distinct events with no cross-thread interactionId bleed), and listener-exception isolation (loop survives a listener that throws on every LlmInvocationEvent)
• LLMOpenAiCostTrackingIT - end-to-end IT with a real OpenAI call
• Full embabel-agent-api suite green: 3530 tests, 0 failures

[igordayen]

looks good! few comments to consider, thanks!

[igordayen]

good, please mark comments as resolved.