feat(ai): per-app AI quota + credit accounting (14.8)#101
Merged
Conversation
Enforce per-app AI budgets at the broker and account every model call
in SQL, closing the 11.8/11.9 residue ("budget enforcement (14.8)").
- account.db v2: `ai_usage` (per-call app/verb/provider/model/tokens/
credits rows) + `ai_credit_ledger` (bundled-credit grants/debits with
a synced flag for the future /v1/usage/ingest reporter). Typed repos
(`AiUsageRepository`, `CreditLedgerRepository`); no SQL outside them.
- Static per-model credit rate table (1 credit = 1 USD list price,
integer micro-credits, $/MTok transcription; conservative fallback so
unknown paid models never meter as free; Ollama = 0).
- `AiQuotaService`: pre-dispatch `checkBudget` over a rolling 30-day
window (tokens and/or credits, per app; null = unlimited) throwing
the DISTINCT `AiBudgetExhausted` wire error (never a generic
Unavailable); fail-closed when a budget exists but the accounting
store is unreadable. Shell callers (`shell`, `_shell.*`) are exempt;
automations meter under the automations app id. Post-call
`recordUsage` prices + persists every call (best-effort) and debits
bundled credits only for platform-billed calls under the
BundledAiCredits entitlement — always BYO today, so no debit fires
until platform routing lands (documented TODO).
- SDK: `AI_BUDGET_EXHAUSTED_ERROR_KIND` (sdk-types) + typed
`AiBudgetExhausted` error class (sdk); the Agent app renders a
dedicated budget-exhausted message.
- app-preload: AI-service rejections re-thrown as clone-safe
`{kind,name,message}` objects — the contextBridge structured clone
collapsed custom Error names to "Error", so no typed error kind
(Unavailable/CapabilityDenied/AiBudgetExhausted) ever actually
reached sandboxed app code (pre-existing gap, found via runtime
verification).
- Settings → AI: usage list is now the 30-day SQL aggregate (calls /
tokens / credits + provider-model slices), budget popover edits token
and credit ceilings, rows show consumed % and a "Budget exhausted"
badge mirroring the broker comparison. Dead JSONL aggregation
removed (the 11.8 log stays as per-device diagnostics).
Verified end-to-end against the built shell (fake Ollama → Agent chat
→ usage row → budget set via panel → next call refused → clear budget
→ unblocked). +42 tests; full suite, typecheck, lint green.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
e98ddd9 to
3f1bca0
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements iteration 14.8 — per-app AI quota + bundled credit accounting: the enforcement/accounting residue the 11.6/11.9 rows left behind ("budget enforcement (14.8) · per-app budgets / routing UI").
What changed
Usage accounting (SQL, per vault)
account.dbv2 migration (storage/account-schema.ts): newai_usagetable (one row per AI broker model call — ts, app, verb, provider/model, prompt/completion/total tokens, cost in micro-credits, outcome, duration; indexed on(app_id, ts)and(ts)) andai_credit_ledger(bundled-credit grants/debits with asyncedflag +remote_ref).AiUsageRepository(main/ai/ai-usage-repo.ts) — typed repo, no SQL outside it:insert,totalsForApp(appId, sinceTs)(the budget-check read),summarizeByApp(sinceTs)(per-app + per provider/model breakdown for the panel),deleteBefore(90-day retention prune).main/ai/model-rates.ts): 1 credit = 1 USD provider list price, stored as integer micro-credits (no float drift in SQL sums). Rates are$/MTok × 1e6— updating a price is pasting the provider's numbers. Longest-prefix model match → provider default → conservative cloud fallback (an unknown paid model never meters as free); Ollama = 0. Per-sideceilso nonzero usage on a paid model never rounds down to free.aggregateAiUsageByApp) was deleted. The panel + enforcement read the SQL substrate.Per-app budget enforcement (fail-closed, distinct error)
AiQuotaService(main/ai/ai-quota.ts) on the broker's AI dispatch path:checkBudget(appId)runs before every model-calling verb (generate/transform/extract; nevercost). Budgets come from the existing 11.9ai-settings.jsonstore (now{maxTokens?, maxCredits?}per app, rolling 30-day window; absent = unlimited). Usage is summed fromai_usage.AiBudgetExhaustedErrorwith wire kindAiBudgetExhausted(the broker mapserror.name→error.kind), never a genericUnavailable.@brainstorm/sdk-typesexportsAI_BUDGET_EXHAUSTED_ERROR_KIND;@brainstorm/sdkreconstructs a typedAiBudgetExhaustederror class so apps caninstanceof. The Agent app now renders a dedicated "used up its 30-day AI budget" message for it.Unavailable, never approval (mirrors the broker's ledger posture). A budget rejection still lands an error provenance row.ai-quota.ts): budgets bind broker-verified app identities. Exempt: the privileged dashboard (shell— only ever registered for the dashboard webContents, unforgeable pastverifyAppIdentity) and the reserved_shell.*main-process namespace (e.g._shell.ai). Automation workflows call AI under the automations app's identity, so they meter against that app's budget like any app call.recordUsageruns after every call (success + failure), prices it, inserts the row, and opportunistically prunes rows older than 90 days (hourly at most). Best-effort like the JSONL sink — accounting can never break a completed call.Bundled-credit accounting seam (14.3/14.6)
CreditLedgerRepository(main/billing/credit-ledger-repo.ts):grant/debitentries (positive integer micro-credits only — a zero/negative row could silently mint credits),balanceMicro()= grants − debits,unsynced()/markSynced()for a future control-plane reporter.isPlatformBilled: () => false— every provider is BYO/local, so no debit is ever written and BYO calls never consume plan credits; routing is untouched. The debit path is fully wired and tested behind the seam: it fires only when (a) the entitlement carriesFeatureFlag.BundledAiCreditsand (b) the call's provider is platform-billed.unsynced()debits to the cloud usage-ingest contract (/v1/usage/ingest— today it metersstorage.bytes/sync.egress.bytesonly; an AI-credits meter is the cross-plane follow-up) and stampsremote_ref.VaultSessiongains cachedaiUsageRepo()/aiCreditLedger()accessors (account.db), nulled on dispose.Settings → AI (existing panel — no Billing panel touched)
ai-settings:usagereturns{windowMs, apps}incl. per provider/model slices and credits; per-call rows never cross IPC; no vault → empty).>=/micro-unit comparison the broker enforces (renderer/settings/ai-budget-view.ts, unit-tested).Found + fixed while verifying: error kinds never survived the app contextBridge
Runtime verification surfaced a pre-existing platform gap: errors thrown by the preload-built SDK runtime cross
contextBridgevia a structured clone that collapses a custom Errornameto"Error"and drops extra fields — so no typed error kind (Unavailable,CapabilityDenied, and nowAiBudgetExhausted) ever actually reached sandboxed app code; the Agent app's kind branches were dead on the non-tool path. Fixed for the AI service (the only sandboxedservices.aiconsumer) inapp-preload.ts: AI-service rejections are re-thrown as plain clone-safe{kind, name, message}objects — exactly thee.kind ?? e.nameshape apps already read. Verified live (below); the same wrapper is the pattern to extend if other services need distinguishable kinds.Verification (real shell, Playwright vs built Electron)
Drove the full path end-to-end with a fake local Ollama reporting 60k/60k tokens per call: Agent app chat → reply renders → Settings → AI shows the accounted usage row (1 call · 120,000 tokens · Last 30 days) → set a 1,000-token budget through the panel popover (probe:
0keeps Set disabled) → row flips to "Budget exhausted" → the next Agent message is refused with the dedicated budget message ("The Agent app has used up its 30-day AI budget…") → clearing the budget through the popover un-blocks the app (next reply renders). Screenshots captured for panel + refusal states. (Throwaway spec, not committed.)Tests
model-rates.test.ts— known-model pricing, longest-prefix, provider/global fallbacks, ceil-never-free, clamping (7)ai-usage-repo.test.ts— v2 migration lands (schema version 2), insert/totals, window rollover, per app+provider/model summary, retention delete (6)credit-ledger-repo.test.ts— balance math, positive-amount guard, unsynced/markSynced, metadata round-trip (6)ai-quota.test.ts— token + credit ceilings block with the distinct error, window rollover frees the budget, per-app isolation, shell/_shell.*exemption, fail-closed on unreadable stores, priced rows, best-effort recording, retention prune, bundled-credit debit seam (BYO never debits, no-entitlement never debits, failures never debit) (16)ai-service.test.ts— gate runs before dispatch with the envelope's app id, blocks all three verbs (provider never called),costnever gated, rejection still records provenance (+4)ai-settings-store.test.ts— credit budgets validated/capped, object-shapedsetAppBudget(+3 updated)sdk errors.test.ts—AiBudgetExhaustedreconstruction, distinct fromUnavailable(new)ai-budget-view.test.ts— credits formatting, exhaustion mirror of the broker comparison, consumed fraction (new)Full suite: 14,377 passed / 0 failed (initial run had 10 env-only failures from the fresh worktree's missing electron
path.txt; green after restoring it).bun run typecheck(packages + apps) andbun run lint(biome + token/reactivity/i18n ratchets) clean.🤖 Generated with Claude Code