Skip to content

[Bug]: Usage always 0 for custom OpenAI-compatible provider when using embedded runner (--local) #96463

Closed

Description

@bazhang1618

Bug type

Behavior bug (incorrect output/state without crash)

Beta release blocker

No

Summary

When using openclaw agent --local --json with a custom provider configured as api: "openai-completions", the lastCallUsage field in the JSON output, as well as usage entries in session files and trajectory logs, are always 0, despite the upstream provider correctly returning prompt_tokens and completion_tokens in its streaming response.

Steps to reproduce

1、Configure a custom provider in ~/.openclaw/openclaw.json with api: "openai-completions" and a valid baseUrl pointing to an OpenAI-compatible endpoint.

2、Run the following command:

bash
openclaw agent --local --json --session-id test-usage --message "Hi"
3、Inspect the meta.agentMeta.lastCallUsage field in the JSON output.

4、Inspect the session file:

bash
cat ~/.openclaw/agents/main/sessions/test-usage.jsonl | grep usage
5、Inspect the trajectory file:

bash
cat ~/.openclaw/agents/main/sessions/test-usage.trajectory.jsonl | grep usage
6、Run openclaw sessions list and observe the Tokens column.

Expected behavior

Based on the upstream provider's documented and observed behavior, the usage fields (input, output, cacheRead, cacheWrite, totalTokens) in the session file, trajectory, and --json output should contain the non-zero token counts returned by the provider, matching the values observed in the raw API response (e.g., {"prompt_tokens":5, "completion_tokens":10, "total_tokens":15}).

Actual behavior

The following observed outputs consistently show zero usage:

lastCallUsage in --json output: {"input":0,"output":0,"cacheRead":0,"cacheWrite":0,"total":0}

assistant message usage in session file: {"input":0,"output":0,"cacheRead":0,"cacheWrite":0,"totalTokens":0}

model.completed event usage in trajectory: {"input":0,"output":0,"cacheRead":0,"cacheWrite":0,"totalTokens":0}

openclaw sessions list shows unknown/200k (?%) in the Tokens column.

OpenClaw version

2026.6.10

Operating system

macOS 15.7.3 (arm64)

Install method

npm global

Model

DeepSeek-V4-Pro (served via custom MaaS platform)

Provider / routing chain

openclaw agent (embedded runner) -> custom-maas-hikvision-com-cn (api: openai-completions) -> MaaS platform

Additional provider/model setup details

json
{
"models": {
"providers": {
"custom-maas-example": {
"baseUrl": "https://maas.example.com/v1",
"api": "openai-completions",
"models": [
{ "id": "DeepSeek-V4-Pro", "contextWindow": 200000 }
]
}
}
}
}
api: "openai-completions" is explicitly set. The provider is a custom name (custom-maas-*).

Logs, screenshots, and evidence

1、Direct API verification: curl to the MaaS endpoint with stream: true and stream_options: {include_usage: true} returns a final chunk containing:

json
{"usage":{"prompt_tokens":5,"completion_tokens":10,"total_tokens":15}}
2、OpenAI SDK verification: A standalone Node.js test using the same openai SDK version (6.26.0) correctly receives chunk.usage with the same non-zero values.

3、Code inspection (OpenClaw 6.10, compiled):

In openai-completions-vbhA-xck.js, the streaming transport layer unconditionally parses chunk.usage:

js
if (chunk.usage) output.usage = parseChunkUsage(chunk.usage, model);
In run-attempt-DRhLt3eF.js, this.tokenUsage is only updated via:

js
case "thread/tokenUsage/updated":
    this.handleTokenUsage(params);
    break;
The "thread/tokenUsage/updated" event is only emitted by the Codex/Responses API path, not by the openai-completions streaming path.

In embedded-agent-BgF2MOkH.js, attempt.attemptUsage falls back to lastAssistantUsage (which reads from the session file, already 0), and mergeUsageIntoAccumulator receives zero.

4、Session file evidence: For the test-usage session, the assistant message in sessions/test-usage.jsonl contains:

json
"usage":{"input":0,"output":0,"cacheRead":0,"cacheWrite":0,"totalTokens":0}

Impact and severity

Affected users/systems/channels: All users of openclaw agent --local --json with a custom provider configured as api: "openai-completions". This includes users of local LLM servers (e.g., llama.cpp, vLLM) and third-party MaaS platforms.

Severity: High. Token usage data is critical for cost tracking, monitoring, auto-compaction, and session management. This bug makes all these features effectively blind.

Frequency: Always reproducible (6/6 observed attempts).

Consequence: Auto-compaction does not trigger, long-running tasks may fail silently due to context overflow, and cost/metering data is entirely missing from the system.

Additional information

This bug does not affect the cli-runner path (non---local), which correctly passes output to buildCliRunResult and preserves usage data.

The root cause is isolated to the embedded-agent runner's failure to bridge the already-parsed output.usage from the streaming transport layer to the attempt object's token usage, as it relies exclusively on the Codex-specific thread/tokenUsage/updated event.

A workaround is to avoid --local mode if the deployment allows it, but this is not feasible for many local/embedded use cases.

The issue was observed on OpenClaw 2026.6.10, but code analysis suggests it likely affects all versions where the embedded runner and openai-completions transport are used together.

Activity

  1. clawsweeper commented on Jun 24, 2026

    @clawsweeper
    Contributor

    Codex review: this still needs some work.

    Summary
    The accounting repair and missing-usage diagnostics are implemented, but the assigned maintainer explicitly retained this issue for a compatibility-policy decision that later discussion does not explicitly settle.

    Reproducibility: Source establishes that an unknown remote endpoint without the capability override is not asked for streaming usage; historical endpoint traces corroborate this path, but the original universal-zero accounting defect is repaired and no current-main live reproduction was run.

    Root-cause cluster
    Relationship: canonical
    Canonical: #96463
    Summary: This issue retains the general remote-endpoint request-policy question; merged work addresses received accounting and missing-usage context handling separately.

    Members:

    Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

    Maintainer decision needed

    • Question: Does the documented opt-in plus merged accounting and estimation work satisfy this issue, or should unknown remote endpoints request streaming usage by default?
    • Recommendation: Accept the existing opt-in policy: Confirm that the documented capability setting and merged repairs satisfy the intended outcome, then manually resolve the retained issue.
    • Why: The assigned maintainer explicitly retained this choice; later evidence verifies supported routes and improved diagnostics without explicitly approving a universal default.

    Next step
    vincentkoc should explicitly accept the documented opt-in policy or define the remaining default-policy change in light of the merged repairs and later endpoint evidence.

    Review details

    Best possible solution:

    Retain verified per-model usage opt-ins and the implemented unavailable-usage guidance unless the assigned maintainer approves a broader compatibility policy.

    Do we have a high-confidence way to reproduce the issue?

    Source establishes that an unknown remote endpoint without the capability override is not asked for streaming usage; historical endpoint traces corroborate this path, but the original universal-zero accounting defect is repaired and no current-main live reproduction was run.

    Is this the best way to solve the issue?

    Another accounting patch would duplicate merged work; retaining the existing opt-in is supported by successful endpoint evidence and the historical compatibility rationale.

    AGENTS.md: found and applied where relevant.

    Remaining risk / open question:

    • Successful usage opt-ins on the reported endpoints do not establish that every unknown remote endpoint accepts stream_options.
    • The assigned maintainer has not explicitly resolved the retained policy question after the diagnostics repair and later retest.

    Codex review notes: model internal, reasoning medium; reviewed against eab05c916735.

    Label changes

    Label changes:

    • remove impact:auth-provider: Current review impact labels are impact:session-state.

    Label justifications:

    • P2: Custom remote usage accounting has a verified configuration path, and current main mitigates missing context measurements through estimation and guidance.
    • impact:session-state: Missing provider measurements affect persisted usage and session context accounting.
    Evidence reviewed

    What I checked:

    How this review workflow works
    • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
    • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
    • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
    • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
    • Maintainers can also comment @clawsweeper review to request a fresh review only.
    • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge; those execution lanes are retired.
    • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

    Reviewed October 16, 2026, 1:23 AM ET / 05:23 UTC.

  2. added
    P2Normal backlog priority with limited blast radius.
    impact:data-lossCan lose, corrupt, or silently drop user/session/config data.
    impact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.
    impact:auth-providerAuth, provider routing, model choice, or SecretRef resolution may break.
    issue-rating: 🐚 platinum hermitGood issue quality with a plausible reproduction path needing some confirmation.
    clawsweeper:needs-live-reproClawSweeper needs live local, crabbox, or manual validation to confirm this issue.
    clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.
    clawsweeper:needs-maintainer-reviewClawSweeper marked this issue as needing maintainer review before automation.
    clawsweeper:needs-product-decisionClawSweeper marked this issue as needing a product or behavior decision.
    on Jun 24, 2026
  3. self-assigned this
    on Jul 2, 2026
  4. vincentkoc commented on Jul 2, 2026

    @vincentkoc
    Member

    The narrow embedded-accounting subset landed in #96523 as cb44f40474bd88d3ba2e1c3c3f60767c9f6194e5.

    That change preserves nonzero usage already received from an OpenAI-compatible stream before the assistant message is persisted, including retry/current-attempt and reasoning-token handling. It does not change generic custom-provider request shaping or enable stream_options.include_usage by default.

    Keeping this issue open for that separate compatibility/product decision. Useful evidence would be a current-main live endpoint run showing whether the provider accepts and returns usage with explicit compat.supportsUsageInStreaming: true, plus whether enabling it has any endpoint-specific failure mode.

  5. kesslerio commented on Jul 14, 2026

    @kesslerio
    Contributor

    @vincentkoc I ran this against my GX10 vLLM endpoint.

    The prod route is a generic custom openai-completions provider running deepseek-v4-flash-dspark. It had no supportsUsageInStreaming override. All 210 assistant records in the affected session saved zero usage:

    {"input":0,"output":0,"cacheRead":0,"cacheWrite":0,"totalTokens":0}

    The endpoint returns usage when OpenClaw asks for it:

    non-streaming:                    prompt_tokens=9 completion_tokens=2 total_tokens=11
    streaming without stream_options: no usage chunk
    streaming with include_usage=true: prompt_tokens=9 completion_tokens=2 total_tokens=11
    

    I then spun up an isolated, no-channel Docker staging state with the same 2026.6.11 (e085fa1) image and this per-model setting:

    "compat": { "supportsUsageInStreaming": true }

    The embedded agent --local call returned HTTP 200. Its metadata and persisted assistant record both had:

    input=22940 output=3 totalTokens=22943
    

    This endpoint supports the explicit setting, and it restores embedded-runner usage accounting. It also explains the misleading status output on the prod route. The 1M context window was correct, but OpenClaw had no provider usage to save, so the overflow diagnostic reported observedTokens=unknown.

  6. 9 remaining items

  7. steipete commented on Oct 11, 2026

    @steipete
    Contributor

    Retested on current main: with local OpenAI-compatible servers (Ollama /v1, LM Studio, llama-server), openclaw agent --local now reports exact usage and context.

    The difference in this report is the endpoint type. OpenClaw only requests streamed usage (stream_options.include_usage) automatically for endpoints it knows support it. For an unknown remote custom endpoint, enable it explicitly with compat.supportsUsageInStreaming: true on the provider. Your direct API comparison sent include_usage, but the OpenClaw config in the report didn't opt in. Since #166151, missing streamed usage is shown as unavailable or estimated rather than a silent 0, and the docs and warnings point to this setting.

    If usage is still 0 with compat.supportsUsageInStreaming: true on a current release, please share the provider config (without keys) and a raw streamed response tail.

  8. added
    clawsweeper:intake:bugPreliminary submission classification; full source review pending.
    and removed
    impact:auth-providerAuth, provider routing, model choice, or SecretRef resolution may break.
    clawsweeper:intake:bugPreliminary submission classification; full source review pending.
    on Oct 16, 2026
  9. ericcurtin commented on Oct 17, 2026

    @ericcurtin
    Contributor

    For a LAN llmman host reached by IP, the flag is documented in #174877. If you have a moment, please try https://github.com/llmmanorg/llmman too. Thanks!

  10. steipete commented on Oct 24, 2026

    @steipete
    Contributor

    Thanks for filing this. We're closing older issues that no longer look actionable on current main.

    The usage accounting repair and missing-usage diagnostics have landed. Unknown remote OpenAI-compatible endpoints only send streaming usage if you opt in through the documented capability override. Please open a fresh issue if usage is still zero with that enabled.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

P2Normal backlog priority with limited blast radius.bugSomething isn't workingbug:behaviorIncorrect behavior without a crashclawsweeper:needs-maintainer-reviewClawSweeper marked this issue as needing maintainer review before automation.clawsweeper:needs-product-decisionClawSweeper marked this issue as needing a product or behavior decision.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.impact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.

Type

No type

Fields

Priority

None yet

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions