fix(api-proxy): track token usage for aborted streaming responses#6517
Conversation
The HTTP token tracker only finalized usage on a clean `proxyRes` 'end' event. Streaming (SSE) clients such as Codex/OpenAI `/responses` routinely tear down the socket after receiving the final event, so the upstream response emits 'aborted'/'close' but never 'end'. In that case `finalizeHttpTracking` never ran and the usage accumulated per-chunk was silently dropped — no `TRACK_END` and no `token-usage.jsonl` record. This is why the `verify_token_usage` smoke check is flaky: on a passing run only ~half the streams finalize (e.g. 7 of 14), and when zero finalize the check fails with "No token-usage records found". Finalize on a premature 'aborted'/'close' as a fallback, guarded so it is a no-op after a clean 'end' (and won't finalize before a decompressor flushes). A once-guard ensures usage is never written twice. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 79b3e12a-b35f-47e3-adb5-c0178e7eb873
There was a problem hiding this comment.
Pull request overview
Adds fallback token-usage finalization for prematurely closed streaming responses.
Changes:
- Handles
abortedandcloseresponse events with once-only finalization. - Adds tests for aborted streams and duplicate finalization.
Show a summary per file
| File | Description |
|---|---|
containers/api-proxy/token-tracker-http.js |
Adds premature-close finalization; compressed streams require correction. |
containers/api-proxy/token-tracker.http.test.js |
Tests aborted and clean-close behavior. |
Review details
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 2/2 changed files
- Comments generated: 1
- Review effort level: Medium
|
| Metric | Base | PR | Delta |
|---|---|---|---|
| Lines | 98.99% | 98.99% | ➡️ +0.00% |
| Statements | 98.92% | 98.91% | 📉 -0.01% |
| Functions | 99.40% | 99.41% | ➡️ +0.01% |
| Branches | 95.34% | 95.30% | 📉 -0.04% |
📁 Per-file Coverage Changes (3 files)
| File | Lines (Before → After) | Statements (Before → After) |
|---|---|---|
src/container-lifecycle.ts |
100.0% → 98.6% (-1.36%) | 100.0% → 98.1% (-1.93%) |
src/commands/main-action.ts |
92.6% → 92.6% (+0.05%) | 92.0% → 92.1% (+0.05%) |
src/log-directory-setup.ts |
96.2% → 100.0% (+3.78%) | 96.3% → 100.0% (+3.71%) |
Coverage comparison generated by scripts/ci/compare-coverage.ts
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
|
📡 Smoke OTel Tracing completed. All tracing scenarios validated. ✅ |
|
✅ Security Guard completed successfully! Security review complete for PR #6517. Changes are in |
|
✅ Smoke Gemini completed. All facets verified. 💎 |
|
✅ Smoke Copilot BYOK AOAI (Entra) completed. Copilot AOAI BYOK (Entra) mode operational. 🔓 |
|
Chroot tests passed! Smoke Chroot - All security and functionality tests succeeded. |
|
✅ Smoke Copilot BYOK AOAI (api-key) completed. Copilot AOAI BYOK (api-key) mode operational. 🔓 |
|
📰 VERDICT: Smoke Copilot has concluded. All systems operational. This is a developing story. 🎤 |
|
✅ Smoke Claude passed |
|
✨ The prophecy is fulfilled... Smoke Codex has completed its mystical journey. The stars align. 🌟 |
|
✅ Contribution Check completed successfully! Contribution check complete for PR #6517: no guideline issues found in the provided context. The PR includes tests for the new behavior, the description explains the problem/root cause/fix clearly, and the files are organized appropriately under containers/api-proxy. |
|
✅ Build Test Suite completed successfully! |
|
🛡️ Smoke Copilot Network Isolation confirmed the egress allowlist is enforced. ✅ |
|
🔑 Smoke Copilot PAT PAT auth validated. All systems operational. ✅ |
|
🔌 Smoke Services — All services reachable! ✅ |
|
✅ Smoke Copilot BYOK completed. Copilot BYOK mode operational. 🔓 |
|
📰 VERDICT: Smoke Docker Sbx has concluded. All systems operational. This is a developing story. 🎤 |
|
@lpcox — Smoke test results for this PR: EGRESS_RESULT allow=pass deny=pass
Overall: PASS Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "example.com"See Network Configuration for more information.
|
github-actions
Bot
added
the
smoke-copilot-network-isolation
Smoke Test: Claude Engine Validation
Overall Result: PASS ✅
|
🔬 Smoke Test: Copilot PAT Auth — PASS
Overall: PASS · Auth mode: PAT (COPILOT_GITHUB_TOKEN) /cc @lpcox
|
🔬 Smoke Test Results
Overall: INCONCLUSIVE — Pre-computed test data ( cc @lpcox
|
Smoke Test: Copilot BYOK (Direct Mode) ✅ PASSTest Results:
All tests passed. Agent credentials properly isolated.
|
Gemini Engine Smoke Test Results
Overall status: FAIL Warning Firewall blocked 1 domainThe following domain was blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "localhost"See Network Configuration for more information.
|
Smoke Test: API Proxy OpenTelemetry Tracing
All scenarios pass.
|
Smoke Test Results
Overall: FAIL —
|
|
@lpcox
|
Chroot Version Comparison Results
Overall: ❌ FAILED — Node.js version mismatch detected (host: v24.18.0, chroot: v22.23.1).
|
🏗️ Build Test Suite Results
Overall: 8/8 ecosystems passed — ✅ PASS
|
|
@lpcox
|
|
🔥 Smoke Test Results — Docker Sbx
Overall:
|
…x /responses) (#6542) * fix(api-proxy): finalize token usage on downstream close Codex/OpenAI /responses SSE clients read until the terminal `response.completed` event, then immediately tear down the DOWNSTREAM socket and end their turn — taking the whole awf sandbox (including the api-proxy) down before the UPSTREAM keep-alive socket (proxyRes) ever emits 'end'/'close'/'aborted'. Because `proxyRes.pipe(res)` does not propagate a downstream close back to the upstream stream, none of the existing finalization triggers fired: audits from Smoke Codex runs show 12 TRACK_START / 0 TRACK_END and an empty token-usage.jsonl, failing verify_token_usage. The aborted-stream fallback added in #6517 only watches proxyRes, which is the wrong stream for this case. Thread the downstream `res` through setupTokenTracking -> trackTokenUsage and add `res.on('close')` as an additional finalization trigger. The usage is already parsed per-chunk before the client bails, so this recovers it. The existing finalizeOnce/endedCleanly guards keep it a no-op on clean completions, so no double counting. Adds regression tests covering downstream-close-without-upstream-end and the clean-end-then-downstream-close no-double-count case. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 6bfb1733-2a26-4202-8db1-b99d2b85dab8 * Guard downstream close finalization until SSE usage is observed --------- Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Copilot-Session: 6bfb1733-2a26-4202-8db1-b99d2b85dab8
Problem
The
verify_token_usagesmoke check is flaky and fails intermittently with:(seen most recently on run 29975349162 for PR #6514, unrelated to that PR's changes.)
Root cause
In
containers/api-proxy/token-tracker-http.js,wireListeners()only finalizes token usage on a cleanproxyRes'end'event. Streaming (SSE) clients such as Codex/OpenAI/responsesroutinely tear down the socket right after receiving the final event, so the upstream response emits'aborted'/'close'but never'end'. When that happens,finalizeHttpTrackingnever runs — noTRACK_ENDaudit entry is written and the usage that was already accumulated per-chunk (state.streamingUsage) is silently dropped.Evidence from the artifacts of the failing run vs. a passing
mainrun:TRACK_STARTTRACK_ENDtoken-usage.jsonlrecordsEven on passing runs only ~half the streams finalize; the check happens to pass because at least one does. When zero finalize, it fails.
Fix
Finalize token tracking on a premature
'aborted'/'close'as a fallback, so aborted SSE streams still record the usage accumulated so far. Guards:endedCleanly— the fallback is a no-op after a clean'end', so we never finalize before a decompressor has flushed on the normal path.finalizeOnce— a once-guard ensures usage is written at most once, even when both'end'and'close'fire.Tests
Added two cases to
token-tracker.http.test.js:'aborted'+'close') without a clean'end'.'close'follows a clean'end'.Full api-proxy suite passes (1491 tests).
Note
This is independent of PR #6514 (gVisor retry logic) — that PR only surfaced the flake. This is a standalone fix to the token tracker + its CI sanity check.