llm-pi-ai drops the reasoning-token breakdown pi-ai already reports

[voidwarriorchan]

mapUsage in @deepseek-ai/dsh-llm-pi-ai does not map pi-ai's Usage.reasoning, so TokenUsage.reasoningTokens is always absent on a pi-ai route even when the provider reports a breakdown.

The README explains the omission as an absence upstream:

pi-ai folds reasoning tokens into output usage; there is no separate reasoning count to map.

The first half is accurate — output already includes those tokens — but pi-ai does expose the breakdown separately:

// @earendil-works/pi-ai dist/types.d.ts:263
/**
 * Reasoning/thinking tokens, when the provider reports them. This is a subset of
 * `output`: `output` already includes these tokens.
 */
reasoning?: number;

It is populated for openai-completions (dist/api/openai-completions.js:1077):

reasoning: rawUsage.completion_tokens_details?.reasoning_tokens || 0,

but lib/index.js:418 maps only four fields:

function mapUsage(usage) {
	return {
		inputTokens: usage.input,
		outputTokens: usage.output,
		...usage.cacheRead > 0 ? { cacheReadTokens: usage.cacheRead } : {},
		...usage.cacheWrite > 0 ? { cacheWriteTokens: usage.cacheWrite } : {}
	};
}

TokenUsage.reasoningTokens exists (dsh-llm/lib/types/types.d.ts:128) and dsh-llm-deepseek maps it (lib/index.js:326-331), so the two adapters disagree about the same endpoint shape.

Reproduction

A local OpenAI-compatible server returning completion_tokens_details.reasoning_tokens, driven through ctx.llm.stream() with the same prompt on each adapter:

llm-pi-ai      (api: openai-completions)  usage = {"inputTokens":26,"outputTokens":63}
llm-deepseek   (same endpoint)            usage = {"inputTokens":56,"outputTokens":62,"reasoningTokens":55}

The endpoint's own response carries the breakdown in both cases:

"completion_tokens_details": {"reasoning_tokens": 42}

Suggested fix

Reasoning is a subset of output, so it rides alongside the output count rather than adding to it:

...usage.reasoning === void 0 ? {} : { reasoningTokens: usage.reasoning }

Tested against 0.1.1-rc.1: reasoningTokens then arrives and stays <= outputTokens.

Filed here rather than as an issue because this repository has Issues disabled.

[drscrewdriver]

you can try https://github.com/drscrewdriver/dsh-thinking-levels+https://github.com/drscrewdriver/dsh-llm-openai-completions to workaround

[franksong2702]

Thanks for reporting this — we independently reached the same finding while developing a DSH model-provider plugin. Adding current-version confirmation and regression-test details here instead of opening a duplicate.

The omission is still present at current master commit c389f96bf3 (0.1.3-alpha.2, tested with pi-ai 0.85.1). This is shared llm-pi-ai usage mapping, not specific to our plugin or one model: it affects routes through this mapper when pi-ai supplies usage.reasoning.

We tested the same minimal fix locally in mapUsage:

...usage.reasoning !== undefined ? { reasoningTokens: usage.reasoning } : {},

The intended behavior is:

• Preserve positive counts and an explicitly reported 0; omit the field when pi-ai's normalized usage.reasoning is undefined.
• Leave output, total, and cache counts unchanged. Reasoning is already included in output; adding it again would double-count.
• Apply consistently to successful and failed terminal events, which share this mapper.

One nuance worth documenting: this preserves pi-ai's normalized usage, not the presence of a field in the original HTTP response. A pi-ai provider may normalize a missing wire reasoning count to zero. This patch should not claim to distinguish those two wire cases.

For an assembled local HTTP regression test, the mock OpenAI-compatible response reports:

{
  "prompt_tokens": 10,
  "completion_tokens": 5,
  "total_tokens": 15,
  "prompt_tokens_details": { "cached_tokens": 4 },
  "completion_tokens_details": { "reasoning_tokens": 3 }
}

After pi-ai and Harness assembly, the patched result is:

{
  "inputTokens": 6,
  "outputTokens": 5,
  "totalTokens": 15,
  "cacheReadTokens": 4,
  "reasoningTokens": 3
}

Local validation of the candidate patch:

• pnpm exec vitest run packages/llm/llm-pi-ai: 313 passed, including absent/zero/positive reasoning counts, success/error terminal events, and the HTTP-to-Loader composition test above.
• DSH_SNAPSHOT=replay pnpm exec vitest run --config vitest.snapshot.config.ts snapshots/session/headless.snapshot.ts -t pi-ai-replay-metadata: 1 passed, 90 skipped, with reasoning usage added to the recorded session fixture to cover headless persistence/replay.
• pnpm exec tsc -b packages/llm/llm-pi-ai/tsconfig.json: passed.

These are keyless local checks, not live-provider, billing, or account-quota validation. The patch is local and unpublished; no external PR is being submitted. The package README's statement that there is no separate reasoning count would also need updating alongside the mapper.

This seems like a small provider-neutral usage-detail fix that can stay independent of any model-specific reasoning-level changes.