Fix token usage extraction for OpenAI Responses API (response.usage) in SSE/WS paths

[Copilot]

Bug Fix

What was the bug?

token-tracker only read OpenAI usage from top-level json.usage, so Responses API completion events (response.completed / response.done) with usage at json.response.usage were ignored. As a result, Codex /v1/responses traffic could complete successfully while effective token accounting stayed at zero.

How did you fix it?

• containers/api-proxy/token-tracker.js

  • SSE extraction (extractUsageFromSseLine)
    • Added explicit handling for response.completed and response.done.
    • Reads tokens from json.response.usage (input_tokens, output_tokens, total_tokens).
    • Preserves existing enrichment behavior for reasoning_tokens and cached prompt tokens.
    • Resolves model from json.response.model when present.
  • Non-streaming JSON extraction (extractUsageFromJson)
    • Added fallback usage source at json.response.usage when top-level json.usage is absent.
    • Keeps existing normalization behavior across Anthropic/OpenAI field variants.

• Targeted tests

  • containers/api-proxy/token-tracker.parsing.test.js
    • Added coverage for SSE response.completed usage extraction.
    • Added coverage for reasoning + cached token extraction from nested Responses usage.
    • Added coverage for non-streaming nested response.usage.
  • containers/api-proxy/token-tracker.http.test.js
    • Added streaming /v1/responses case with response.completed.
  • containers/api-proxy/token-tracker.websocket.test.js
    • Added WebSocket response.completed frame case.

Example

if (
  (json.type === "response.completed" || json.type === "response.done") &&
  json.response?.usage
) {
  const u = json.response.usage;
  result.usage = {
    input_tokens: u.input_tokens,
    output_tokens: u.output_tokens,
    total_tokens: u.total_tokens,
  };
  result.model = json.response.model || result.model;
}

Testing

• Added focused parser, HTTP streaming, and WebSocket tests for OpenAI Responses API nested usage (response.usage) and related reasoning/cache fields.

[Copilot]

Pull request overview

Fixes token usage extraction for OpenAI Responses API, where usage data is nested at json.response.usage in response.completed/response.done events rather than at top-level json.usage. Without this fix, Codex /v1/responses traffic completed with zero token accounting.

Changes:

• Added SSE handler for response.completed/response.done events that reads tokens from json.response.usage, including reasoning and cached token enrichment.
• Added fallback to json.response.usage/json.response.model in non-streaming JSON extraction.
• Added parser, HTTP streaming, and WebSocket tests covering nested Responses API usage.

Show a summary per file

File	Description
containers/api-proxy/token-tracker.js	Source the usage object from json.response.usage (and model from json.response.model) as fallback in JSON path; add dedicated SSE branch for response.completed/response.done.
containers/api-proxy/token-tracker.parsing.test.js	New unit tests for nested response.usage in JSON and SSE paths, including reasoning/cached token extraction.
containers/api-proxy/token-tracker.http.test.js	Streaming /v1/responses test asserting input/output token metrics.
containers/api-proxy/token-tracker.websocket.test.js	WebSocket frame test for response.completed usage extraction.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

• Files reviewed: 4/4 changed files
• Comments generated: 0

[github-actions]

🔬 Smoke Test Results

Test	Status
GitHub MCP connectivity	❌ 401 Bad credentials
GitHub.com HTTP connectivity	⚠️ Template vars unexpanded
File write/read	⚠️ Template vars unexpanded

Overall: FAIL — Workflow template variables (${{ steps.smoke-data.outputs.* }}) were not substituted before reaching the agent. GitHub MCP authentication also unavailable (401).

cc @Copilot

📰 BREAKING: Report filed by Smoke Copilot

[github-actions]

Smoke Test Results

• ❌ GitHub API: Failed (network/auth - HTTP 401, firewall blocking external domains)
• ❌ Playwright: Failed (504 Gateway Time-out on github.com)
• ✅ File verify: Passed (smoke-test file readable)

Result: FAIL (1/3 tests passed)

Network connectivity to external domains appears blocked by the sandboxed firewall environment.

💥 [THE END] — Illustrated by Smoke Claude

[github-actions]

Chroot Version Comparison Results

Runtime	Host Version	Chroot Version	Match?
Python	Python 3.12.13	Python 3.12.3	❌
Node.js	v24.15.0	v20.20.2	❌
Go	go1.22.12	go1.22.12	✅

Overall: Not all tests passed — Python and Node.js versions differ between host and chroot environments.

Tested by Smoke Chroot

[github-actions]

🏗️ Build Test Suite Results

Ecosystem	Project	Build/Install	Tests	Status
Bun	elysia	✅	1/1 passed	✅ PASS
Bun	hono	✅	1/1 passed	✅ PASS
C++	fmt	✅	N/A	✅ PASS
C++	json	✅	N/A	✅ PASS
Deno	oak	N/A	1/1 passed	✅ PASS
Deno	std	N/A	1/1 passed	✅ PASS
.NET	hello-world	✅	N/A	✅ PASS
.NET	json-parse	✅	N/A	✅ PASS
Go	color	✅	passed	✅ PASS
Go	env	✅	passed	✅ PASS
Go	uuid	✅	passed	✅ PASS
Java	gson	✅	1/1 passed	✅ PASS
Java	caffeine	✅	1/1 passed	✅ PASS
Node.js	clsx	✅	passed	✅ PASS
Node.js	execa	✅	passed	✅ PASS
Node.js	p-limit	✅	passed	✅ PASS
Rust	fd	✅	1/1 passed	✅ PASS
Rust	zoxide	✅	1/1 passed	✅ PASS

Overall: 8/8 ecosystems passed — ✅ PASS

Generated by Build Test Suite for issue #3178 · ● 4.6M · ◷

[github-actions]

Smoke Test

✅ PR review: Refactor option parser utilities into domain-focused modules with compatibility wrappers; Refactor API proxy request path into focused guard and transport modules
❌ safeinputs-gh PR query: tool unavailable
✅ Playwright: GitHub title verified
❌ Tavily search: no search tool exposed
✅ File/Bash: smoke file verified
❌ Discussion: query tool unavailable
✅ Build: npm ci && npm run build
Overall status: FAIL

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• registry.npmjs.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "registry.npmjs.org"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex

[github-actions]

Smoke Test Results: FAIL (GitHub MCP tool missing, Connectivity SSL error 35)

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• localhost

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "localhost"

See Network Configuration for more information.

💎 Faceted by Smoke Gemini

[github-actions]

Smoke Test Results

Check	Result
Redis PING	❌ Timeout (host.docker.internal unreachable)
PostgreSQL pg_isready	❌ No response
PostgreSQL SELECT 1	❌ No response

Overall: FAIL — host.docker.internal is not reachable from this environment. Service containers may not be running or the host alias is not resolvable.

🔌 Service connectivity validated by Smoke Services

[lpcox]

@copilot fix this failing ci check https://github.com/github/gh-aw-firewall/actions/runs/25897362213/job/76114390532?pr=3178

[Copilot]

@copilot fix this failing ci check https://github.com/github/gh-aw-firewall/actions/runs/25897362213/job/76114390532?pr=3178

I investigated this run. The failure is not from this PR’s code changes: job 76114390532 failed in step Install GitHub Copilot CLI with repeated curl: (22) ... 504 while downloading https://github.com/github/copilot-cli/releases/download/v1.0.40/SHA256SUMS.txt. That’s an external transient download failure before the workflow reached repo build/test steps. Please rerun Smoke Copilot BYOK; no repository code change is needed for this failure mode.

[github-actions]

🔥 Smoke Test: Copilot BYOK (Offline) Mode

Test	Result
GitHub MCP connectivity	❌ (401 auth error)
GitHub.com HTTP connectivity	✅ (HTTP 200)
File write/read	✅ (BYOK smoke test passed)
BYOK inference (api-proxy → api.githubcopilot.com)	✅

Running in BYOK offline mode (COPILOT_OFFLINE=true) via api-proxy → api.githubcopilot.com.

Overall: FAIL (GitHub MCP auth unavailable in this environment)

🔑 BYOK report filed by Smoke Copilot BYOK