Skip to content

fix(openrouter): Avoid double-counting callModel metrics#2005

Merged
Luca Forstner (lforst) merged 6 commits into
mainfrom
fix-openrouter-callmodel-metric-double-count
May 18, 2026
Merged

fix(openrouter): Avoid double-counting callModel metrics#2005
Luca Forstner (lforst) merged 6 commits into
mainfrom
fix-openrouter-callmodel-metric-double-count

Conversation

@ronaldkohhh
Copy link
Copy Markdown
Contributor

@ronaldkohhh Ronald Koh (ronaldkohhh) commented May 14, 2026

Summary

  • Stop logging aggregate token/cost metrics on OpenRouter callModel parent spans when child turn spans were emitted.
  • Preserve parent output and metadata, including turn_count, while leaving per-turn usage on openrouter.beta.responses.send spans.
  • Apply the same behavior to both OpenRouter and OpenRouter Agent instrumentation.

Why

OpenRouter callModel traces currently include usage metrics on both the aggregate parent span and the nested per-turn LLM spans. Span-level token and cost monitors then sum both levels and can report roughly double the actual usage for single-turn calls.

When no child turn spans are emitted, the parent still records aggregate metrics so usage is not dropped for simpler getResponse() flows.

Validation

  • pnpm exec prettier --write js/src/instrumentation/plugins/openrouter-agent-plugin.ts js/src/instrumentation/plugins/openrouter-plugin.ts js/src/instrumentation/plugins/openrouter-agent-plugin.test.ts js/src/instrumentation/plugins/openrouter-plugin.test.ts
  • pnpm exec vitest run src/instrumentation/plugins/openrouter-agent-plugin.test.ts src/instrumentation/plugins/openrouter-plugin.test.ts
  • pnpm run check:typings

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6c954e3026

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "Codex (@codex) review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "Codex (@codex) address that feedback".

Comment thread js/src/instrumentation/plugins/openrouter-plugin.ts
@lforst Luca Forstner (lforst) changed the title Avoid double-counting OpenRouter callModel metrics fix(openrouter): Avoid double-counting callModel metrics May 18, 2026
@lforst Luca Forstner (lforst) merged commit e2d66a3 into main May 18, 2026
46 checks passed
@lforst Luca Forstner (lforst) deleted the fix-openrouter-callmodel-metric-double-count branch May 18, 2026 10:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants