Bug: pi-ai adapter misclassifies 401/403 in error text as AUTH, surfacing misleading "API key is invalid"

[heming-gmh]

Reported via GitHub Discussions per CONTRIBUTING (external PRs not accepted at this stage). Includes a suggested patch for reference and a Chinese translation at the end. All examples are synthetic — no credentials or provider/deployment-specific values are included.

Summary

The pi-ai LLM adapter (@deepseek-ai/dsh-llm-pi-ai) classifies a provider error as an authentication failure (code: "AUTH") whenever the flattened error message text contains a standalone 401 or 403. The client runtime then renders every AUTH failure as the fixed string "API key is invalid" (@deepseek-ai/dsh-client-runtime, displayFailureMessage).

As a result, non-authentication failures — most notably HTTP 403 authorization/permission errors (permission denied, region/policy restrictions, upstream "forbidden" / "Request not allowed"), and any error whose message merely contains the digits 401/403 in an unrelated field (request id, nested raw body, routing metadata, counters) — are all shown to the user as "API key is invalid".

This actively misleads users: it points them at their API key when the key is valid and the real cause is something else (authorization, provider routing, policy). The true diagnostic is preserved in the session log, but the surfaced copy sends users down the wrong path (e.g. rotating a perfectly valid key).

Where

• Classifier: packages/llm/llm-pi-ai/src/stream.ts — classifyPiAiError()

  function classifyPiAiError(message: string): string {
    if (/\b(?:401|403)\b/.test(message)) return 'AUTH'   // <-- 401 and 403 both -> AUTH, matched anywhere in text
    ...
  }

• Display: packages/client/runtime — displayFailureMessage()

  if (record.code === 'AUTH') return 'API key is invalid'

Two distinct problems

1. Semantic: 403 is not authentication. HTTP 401 = unauthenticated (bad/missing credentials). HTTP 403 = authenticated but not authorized (permission, policy, region, model-not-allowed). Collapsing 403 into AUTH — and then into "API key is invalid" — is wrong for the large class of 403s that have nothing to do with the key. The sibling adapter @deepseek-ai/dsh-llm-deepseek (httpErrorCode()) already keys off the structured integer status, but even there 401 || 403 -> AUTH conflates authz with authn.

2. Fragility: matching digits in free text. /\b(?:401|403)\b/ runs against the whole flattened message, so a 401/403 appearing in a request id, a nested provider raw body, routing metadata, or a numeric counter is enough to force an AUTH classification even when the response was not a 401/403 at all. The file's own header comment already documents that pi-ai flattens the error to a bare string and that classification is stuck pattern-matching text; this report is about tightening that text heuristic where it is demonstrably too broad.

Reproduction (synthetic, no real data)

Feed classifyPiAiError a message that represents a 403 permission failure (not an auth failure). A representative synthetic message:

403 {"type":"error","error":{"type":"permission_error","error_type":"permission_denied"},"request_id":"<request-id>","metadata":{"raw":"{\"error\":{\"type\":\"forbidden\",\"message\":\"Request not allowed\"}}"}}

Observed: classifyPiAiError(msg) === "AUTH" -> UI shows "API key is invalid".
Expected: a 403 permission/authorization failure should NOT be reported as an invalid API key; the key was valid.

A second, purely-incidental case (no real HTTP 401/403 involved at all):

500 internal error (trace 403912...)

Observed: "AUTH" (the standalone 403 inside the trace id matches).
Expected: "SERVER".

Suggested fix direction

Within the existing text-matching constraint (no upstream structured error yet):

1. Separate 401 (authn) from 403 (authz). Only treat a leading/standalone 401 as AUTH. Route 403 to a distinct code (e.g. FORBIDDEN / PERMISSION) rather than AUTH.
2. Anchor the status match to a status position, not "anywhere in the message" (e.g. match a leading status token, or require it to sit next to auth-shaped wording), so digits inside ids / nested bodies / counters do not trigger AUTH.
3. Soften the display copy in displayFailureMessage: for authorization failures use neutral copy such as "Authentication or authorization failed (see session log)" instead of the key-specific "API key is invalid", so a valid key is never wrongly implicated.

A concrete patch implementing (1) and (2) in classifyPiAiError, plus (3) in displayFailureMessage, is attached separately.

Notes

• All examples above are synthetic; no credentials, ids, or provider/deployment-specific values are included.
• Verified end to end: the surfaced "API key is invalid" occurred while the API key was actually valid; the underlying failure was a 403 authorization/permission error from the upstream, not an authentication problem.

---

Suggested patch (for reference)

Click to expand the suggested diff

# Suggested patch (for reference / discussion — official repo currently does not accept external PRs)
#
# Targets:
#   1) packages/llm/llm-pi-ai/src/stream.ts   -> classifyPiAiError()
#   2) packages/client/runtime  (failure-display) -> displayFailureMessage()
#
# Goal:
#   - Stop reporting 403 (authorization) as AUTH ("API key is invalid").
#   - Stop matching a bare 401/403 that merely appears anywhere in the flattened
#     message text (request ids, nested raw bodies, routing metadata, counters).
#   - Only a leading/status-position 401 is treated as an authentication failure.
#   - 403 is routed to a distinct FORBIDDEN code.
#   - Display copy no longer hard-implicates the API key for authz failures.
#
# NOTE: This works within the existing text-matching constraint the file header
# already documents (pi-ai flattens the caught error to a bare string). If pi-ai
# later forwards the structured Error/status, classify on that instead.

--- a/packages/llm/llm-pi-ai/src/stream.ts
+++ b/packages/llm/llm-pi-ai/src/stream.ts
@@
 function classifyPiAiError(message: string): string {
-  if (/\b(?:401|403)\b/.test(message)) return 'AUTH'
+  // Anchor the HTTP status to the front of the flattened message (pi-ai prefixes
+  // the status, e.g. "401 {...}" / "403 {...}"), so a 401/403 that merely appears
+  // inside a request id, a nested provider "raw" body, routing metadata, or a
+  // numeric counter no longer forces an auth/forbidden classification.
+  const leadingStatus = /^\s*(\d{3})\b/.exec(message)?.[1]
+  // 401 = unauthenticated (bad/missing credentials) -> AUTH.
+  // 403 = authenticated but not authorized (permission/policy/region) -> FORBIDDEN,
+  // which must NOT be surfaced as "API key is invalid".
+  if (leadingStatus === '401') return 'AUTH'
+  if (leadingStatus === '403') return 'FORBIDDEN'
   if (isQuotaExceededError(message)) return QUOTA_EXCEEDED_CODE
-  if (/\b429\b|rate.?limit/i.test(message)) return 'RATE_LIMIT'
-  if (/\b400\b|invalid.?request/i.test(message)) return 'INVALID_REQUEST'
-  if (/\b5\d\d\b/.test(message)) return 'SERVER'
+  if (leadingStatus === '429' || /rate.?limit/i.test(message)) return 'RATE_LIMIT'
+  if (leadingStatus === '400' || /invalid.?request/i.test(message)) return 'INVALID_REQUEST'
+  if (leadingStatus !== undefined && /^5\d\d$/.test(leadingStatus)) return 'SERVER'
   if (/\btime(?:d)?\s*out\b|timeout/i.test(message)) return 'TIMEOUT'
   if (/stream ended (?:before|without)\b/i.test(message)) return 'TRANSPORT'
   if (/\b(?:network|connection|socket|fetch)\b|\bECONN[A-Z]+\b/i.test(message)
     || /\b(?:other side closed|HTTP2 request did not get a response|WebSocket closed unexpectedly)\b/i.test(message)
     || /\bterminated\b|premature close/i.test(message)) {
     return 'TRANSPORT'
   }
   return 'PI_AI_ERROR'
 }

--- a/packages/client/runtime/src/.../failure-display.ts
+++ b/packages/client/runtime/src/.../failure-display.ts
@@
 function displayFailureMessage(failure: unknown): string {
   if (failure === null || typeof failure !== 'object') return String(failure)
   const record = failure as Record<string, unknown>
-  if (record.code === 'AUTH') return 'API key is invalid'
+  // 401 authentication failure: the credential itself is the likely cause.
+  if (record.code === 'AUTH') return 'API key is invalid'
+  // 403 authorization failure: the key may be perfectly valid; the request was
+  // refused (permission / policy / region / model-not-allowed). Do not implicate
+  // the key. Raw diagnostic remains in the session log.
+  if (record.code === 'FORBIDDEN') return 'Request was forbidden (see session log)'
   return typeof record.message === 'string' ? record.message : JSON.stringify(failure)
 }

---

中文版 (Chinese translation)

概述

pi-ai LLM adapter（@deepseek-ai/dsh-llm-pi-ai）在对错误分类时，只要被拍平的错误消息文本中出现独立的 401 或 403，就会把该错误归类为鉴权失败（code: "AUTH"）。随后 client runtime（@deepseek-ai/dsh-client-runtime 的 displayFailureMessage）会把所有 AUTH 失败统一渲染为固定文案 "API key is invalid"。

后果是：非鉴权类的失败——尤其是 HTTP 403 授权/权限错误（权限不足、地域/策略限制、上游 "forbidden" / "Request not allowed"），以及任何消息里只是恰好包含数字 401/403（出现在 request id、嵌套的 raw body、路由 metadata、计数器等无关字段）的错误——全部会被展示给用户为 "API key is invalid"。

这会主动误导用户：在 API key 明明有效、真实原因是别的（授权、上游路由、策略）时，把矛头指向 key。真实诊断信息虽然保留在 session log 中，但展示出来的文案会把用户引向错误方向（例如去轮换一个完全正常的 key）。

位置

• 分类层：packages/llm/llm-pi-ai/src/stream.ts — classifyPiAiError()

  function classifyPiAiError(message: string): string {
    if (/\b(?:401|403)\b/.test(message)) return 'AUTH'   // <-- 401 和 403 都 -> AUTH，且在文本任意位置匹配
    ...
  }

• 展示层：packages/client/runtime — displayFailureMessage()

  if (record.code === 'AUTH') return 'API key is invalid'

两个层面的问题

1. 语义问题：403 不是鉴权失败。 HTTP 401 = 未认证（凭证错误/缺失）；HTTP 403 = 已认证但未授权（权限、策略、地域、模型不可用）。把 403 塞进 AUTH、再变成 "API key is invalid"，对那一大类与 key 无关的 403 来说是错误的。兄弟 adapter @deepseek-ai/dsh-llm-deepseek（httpErrorCode()）已经基于结构化整型 status 判断，但即便如此，401 || 403 -> AUTH 仍然把授权（authz）与认证（authn）混为一谈。

2. 脆弱性：在自由文本里匹配数字。 /\b(?:401|403)\b/ 是对整条拍平后的消息做匹配，因此只要 401/403 出现在 request id、嵌套的上游 raw body、路由 metadata 或某个数字计数里，就足以强制归类为 AUTH——哪怕这个响应根本不是 401/403。该文件自己的头部注释已经说明 pi-ai 会把错误拍平成裸字符串、分类只能做文本匹配；本报告要处理的正是这个文本启发式中明显过宽的部分。

复现（合成数据，无任何真实信息）

给 classifyPiAiError 喂一条代表 403 权限失败（而非鉴权失败）的消息。一个有代表性的合成消息：

403 {"type":"error","error":{"type":"permission_error","error_type":"permission_denied"},"request_id":"<request-id>","metadata":{"raw":"{\"error\":{\"type\":\"forbidden\",\"message\":\"Request not allowed\"}}"}}

观察到：classifyPiAiError(msg) === "AUTH" -> UI 显示 "API key is invalid"。
期望：403 权限/授权失败不应被报告为 API key 无效；key 是有效的。

第二个纯属误伤的场景（完全不涉及真实的 HTTP 401/403）：

500 internal error (trace 403912...)

观察到："AUTH"（trace id 里独立的 403 命中了）。
期望："SERVER"。

建议的修复方向

在现有文本匹配的约束下（上游暂时不提供结构化错误）：

1. 把 401（认证）与 403（授权）分开。 只把开头/独立位置的 401 判为 AUTH；把 403 路由到独立的 code（如 FORBIDDEN / PERMISSION）而非 AUTH。
2. 把状态码匹配锚定到状态码位置，而非"消息任意位置"（例如匹配开头的状态码 token，或要求它紧邻鉴权语义的措辞），这样 id / 嵌套 body / 计数器里的数字就不会触发 AUTH。
3. 软化展示文案（displayFailureMessage）：对授权类失败使用中性文案，如 "Authentication or authorization failed (see session log)"，而不是针对 key 的 "API key is invalid"，从而避免误伤一个有效的 key。

一份实现 (1)(2) 于 classifyPiAiError、以及 (3) 于 displayFailureMessage 的具体 patch 另附。

说明

• 以上示例均为合成数据；不含任何凭证、id 或与具体 provider / 部署相关的值。
• 已端到端验证：出现 "API key is invalid" 时，API key 实际是有效的；底层失败是来自上游的 403 授权/权限错误，而非认证问题。

[MwumLi]

原因参看 [Bug] HTTP 403 授权拒绝被归类为 AUTH，UI 显示 "API key is invalid"，原始错误不可见 #3222
可以等官方解决 [Feature] 支持统一配置白标 User-Agent（appIdentity），对所有 provider 生效 #3225
临时应对看 dsh-fetch-header-rewrite —— 可配置的请求头改写插件（User-Agent 等）

[heming-gmh]

补充一点时间线事实，供大家参考判断：

• 本贴（Bug: pi-ai adapter misclassifies 401/403 in error text as AUTH, surfacing misleading "API key is invalid" #3073）发布于 2026-08-18 09:05 UTC，正文已包含完整的根因分析（401 vs 403 语义混淆、正则在自由文本中的误匹配问题）和具体的修复 diff（含 classifyPiAiError / displayFailureMessage 补丁）。
• 上面回复中引用的三个讨论——[Bug] HTTP 403 授权拒绝被归类为 AUTH，UI 显示 "API key is invalid"，原始错误不可见 #3222（创建于 2026-08-19 01:03 UTC）、[Feature] 支持统一配置白标 User-Agent（appIdentity），对所有 provider 生效 #3225（01:08 UTC）、dsh-fetch-header-rewrite —— 可配置的请求头改写插件（User-Agent 等） #3227（01:22 UTC）——均由 @MwumLi 在本贴发布约 16 小时之后、在不到 20 分钟内连续新建，且三帖正文均未引用或提及本贴。
• @MwumLi 的 authorAssociation 是 NONE，和我一样是外部贡献者，并非 DeepSeek 官方团队成员，回复内容不代表官方结论。
• dsh-fetch-header-rewrite —— 可配置的请求头改写插件（User-Agent 等） #3227 同时是 @MwumLi 本人维护的第三方 npm 插件 dsh-fetch-header-rewrite 的推广帖（附带其自己的 GitHub 仓库和 npm 包链接）。

以上纯粹陈述可验证的公开时间线和账号信息，不针对个人做其他评价。希望本问题的讨论能准确标注贡献顺序，也期待 DeepSeek 官方维护者对本贴（root cause + 建议 patch）给出正式回应。