Skip to content

feat(ai): error taxonomy + fail-fast retry classification (Phase 1 · Batch 1.1)#119

Merged
qnbs merged 3 commits into
mainfrom
phase1/ai-error-taxonomy
Jun 13, 2026
Merged

feat(ai): error taxonomy + fail-fast retry classification (Phase 1 · Batch 1.1)#119
qnbs merged 3 commits into
mainfrom
phase1/ai-error-taxonomy

Conversation

@qnbs

@qnbs qnbs commented Jun 13, 2026

Copy link
Copy Markdown
Owner

User description

Context

Phase 1 (Reliability/Observability) of the v1.23 audit/perfection engagement, first batch.
Recon corrected the sub-baseline: the Local-AI/Voice low-end stack is already built; the real
gap is AI error handlingwithTransientRetry retried every error (a bad API key, a
policy block, a 4xx) with exponential backoff.

What this PR does (backend + tests only)

  • services/ai/aiErrorTaxonomy.ts — pure classifyAiError(err){ category, retryable, messageKey } across transient | rateLimit | auth | network | offline | policy | invalidRequest | permanent. Classifies by HTTP status, by message markers (reuses the aiPolicy.ts "blocked" strings), and short-circuits on navigator.onLine === false. Unknown errors default to retryable transient (preserves prior behavior).
  • services/ai/aiRetry.tswithTransientRetry now fails fast on non-retryable categories (invalid key / policy / malformed request / offline) instead of backing off a doomed call. Transient/rate-limit/network still retry and honor Retry-After. Emits one structured ai.retry log line per decision with a per-call correlation id (no payloads or keys — respects sanitizeLogContext). New optional shouldRetry lets callers override.
  • Tests (+33): aiErrorTaxonomy.test.ts (table-driven over every category + offline short-circuit + status-beats-message); fail-fast + shouldRetry-override cases in aiRetry.test.ts; logger stubbed in the 3 retry test files for deterministic output.
  • Docs: living plan batch log + CHANGELOG.md [Unreleased].

Scope notes

  • Backward compatible: the two consumers (aiProviderService.ts:437,471) keep working; the new option is optional and unknown errors still retry.
  • messageKey UI/i18n mapping deferred to Batch 1.2 (avoids 11-locale churn here).
  • FU-1 (plugin-worker guard restoration) split out of this batch after a timeboxed probe: the Function.prototype.constructor leak is real but its mechanism is subtler than a symmetric defineProperty fix; it's benign (the worker compiles via the captured GlobalFunction) and is documented in docs/AUDIT-PERFECTION-PLAN-v1.23.md for a dedicated fix.

Verification

  • pnpm exec vitest run taxonomy + all retry suites + plugin.worker → 90 passed.
  • pnpm run lint (1312 files) + pnpm run typecheck → clean.
  • git diff --name-only → only the taxonomy, retry, 3 retry tests, and 2 docs (no production source beyond aiRetry).

🤖 Generated with Claude Code


CodeAnt-AI Description

Stop retrying AI calls that cannot succeed and add clear error classification

What Changed

  • AI errors are now grouped into clear categories so retry handling can tell apart temporary failures, rate limits, network problems, auth issues, policy blocks, invalid requests, and offline states
  • Retry now stops immediately for non-retryable errors like invalid API keys, policy blocks, malformed requests, and offline mode instead of waiting and trying again
  • Retryable AI failures still back off and keep honoring server-provided retry delays
  • Each retry decision now produces a structured log entry, and callers can override the default retry decision when needed
  • Added tests for error classification, fail-fast behavior, custom retry overrides, and deterministic retry logging

Impact

✅ Fewer repeated failed AI calls
✅ Faster errors for bad keys, policy blocks, and offline users
✅ Clearer AI failure handling

💡 Usage Guide

Checking Your Pull Request

Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.

Talking to CodeAnt AI

Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:

@codeant-ai ask: Your question here

This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.

Example

@codeant-ai ask: Can you suggest a safer alternative to storing this secret?

Preserve Org Learnings with CodeAnt

You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:

@codeant-ai: Your feedback here

This helps CodeAnt AI learn and adapt to your team's coding style and standards.

Example

@codeant-ai: Do not flag unused imports.

Retrigger review

Ask CodeAnt AI to review the PR again, by typing:

@codeant-ai: review

Check Your Repository Health

To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.

Phase 1 · Batch 1.1 (Master Prompt v1.1 engagement).

- services/ai/aiErrorTaxonomy.ts: pure classifyAiError(err) ->
  {category, retryable, messageKey} across transient/rateLimit/auth/network/
  offline/policy/invalidRequest/permanent (HTTP status + aiPolicy 'blocked'
  markers + offline short-circuit). Table-driven tests (26).
- services/ai/aiRetry.ts: withTransientRetry now fails fast on non-retryable
  errors (auth/policy/invalid-request/offline) via a shouldRetry default,
  instead of backing off a doomed call. Transient/rateLimit/network still retry
  (Retry-After honored). One structured ai.retry log line per decision with a
  per-call correlation id (no payloads/keys). Backward compatible.
- Tests: fail-fast cases + custom shouldRetry override; logger stubbed in the
  3 retry test files for deterministic output.
- Docs: living plan batch log + CHANGELOG [Unreleased].

FU-1 (plugin-worker guard restoration) SPLIT from this batch after a timeboxed
probe: the Function.prototype.constructor leak is real but its mechanism is
subtler than a symmetric defineProperty fix; documented in the living plan for a
dedicated fix. Benign (worker compiles via captured GlobalFunction).

i18n/UI mapping of messageKey deferred to Batch 1.2.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@codeant-ai

codeant-ai Bot commented Jun 13, 2026

Copy link
Copy Markdown

CodeAnt AI is reviewing your PR.


Thanks for using CodeAnt! 🎉

We're free for open-source projects. if you're enjoying it, help us grow by sharing.

Share on X ·
Reddit ·
LinkedIn

@vercel

vercel Bot commented Jun 13, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
storycraft-studio Ready Ready Preview, Comment Jun 13, 2026 11:59am

@codeant-ai codeant-ai Bot added the size:L This PR changes 100-499 lines, ignoring generated files label Jun 13, 2026
Comment thread tests/unit/services/aiErrorTaxonomy.test.ts Outdated
CodeAnt (Minor): avoid embedding a real service hostname in test text.
api.openai.com -> api.example.invalid (RFC 2606/6761 reserved). The hostname is
not asserted on — classification keys off the ENOTFOUND marker — so behavior is
unchanged; 26/26 still green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Comment thread services/ai/aiErrorTaxonomy.ts Outdated
@codeant-ai

codeant-ai Bot commented Jun 13, 2026

Copy link
Copy Markdown

CodeAnt AI finished reviewing your PR.

CodeAnt: classifying messages containing 'aborted' as transient made
user/timeout AbortController cancellations retryable. Add a non-retryable
'canceled' category: detect DOMException AbortError (name/code 20) and
abort/cancel messages; remove 'aborted' from the transient regex. +4 tests
(30 taxonomy / 58 total green).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@qnbs
qnbs merged commit 4208e43 into main Jun 13, 2026
17 checks passed
@qnbs
qnbs deleted the phase1/ai-error-taxonomy branch June 13, 2026 12:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L This PR changes 100-499 lines, ignoring generated files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant