feat(ai): usage + cost attribution to an admin-only, tamper-locked database#113
Merged
Conversation
…+ editable pricing (C1) Log token + cost attribution for every server-side model request into a server-managed, admin-only OpenBook database, with admin-editable pricing. - providers.ts: surface real token counts via a new onUsage(TokenUsage) callback — AnthropicEngine reads message_start/message_delta usage frames (incl. cache tokens), OpenAiCompatEngine (+MlxEngine) sets stream_options.include_usage and reads the trailing chunk, LlamaEngine best-effort tokenize, MockEngine deterministic counts. - ai/usage.ts (new): AiUsageLog — idempotent seed of the usage database on a restricted host page (owner/admin/ACL read only) marked managed + 30-day auto-expiry; DEFAULT_PRICING (Claude + common OpenAI models, local = 0) with an admin override in settings.aiPricing; effective = override→default→null; cost snapshotted at write time; one row per model call attributed to the server principal (never a client id, never the API key). - routes.ts / agent.ts: log generate / complete / each agent turn. - app.ts: reject end-user create-row/update-row/patch/delete/reorder against the managed usage database (server writes bypass via the store). - routes.ts: GET/PUT /api/ai/pricing + PUT /api/ai/usage/retention, admin-gated. - store.ts: generic getSetting/setSetting. sdk: pricing types + client methods + DatabaseSchema.managed marker. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X78XTAemKrFw1uJpDpCEaG
11 tests covering: MockEngine deterministic usage; one attributed row per agent
turn / generate call (known model → DEFAULT_PRICING cost, unknown → null cost,
server principal only, no key leak); restricted host page (owner/admin read,
viewer/guest 404); managed-DB write rejection (403 for owner/admin, 404 for
guest) with server writes still landing; admin pricing GET/PUT (403 for
non-admin) changing the next row's effective cost; seeded autoExpiry
{enabled,days:30,basis:created} + managed marker + admin retention edit;
idempotent seed (no duplicate DB).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01X78XTAemKrFw1uJpDpCEaG
… restricted + parser tests + pricing validation The AI usage-attribution DB (C1) is server-managed and admin-only, with write-immutability enforced by rejectManaged() on the /api/databases/* routes. Its host page and attribution rows are ordinary pages, so the invariant leaked through the generic /api/pages/* routes: an owner/admin could delete rows, trash the host, un-restrict it (exposing every user's usage), re-home it, or grant it an ACL. - Track the usage DB's host page id in AiUsageLog and add isManagedPage() (true for the host page or any row whose database_id is the usage DB). - Add rejectManagedPage() in app.ts, called AFTER the access gate on PATCH :id, PATCH :id/properties, DELETE :id, PUT :id/move, PUT :id/visibility, POST/DELETE :id/acl — a non-reader still gets an existence-hiding 404, only a would-be mutator sees the managed 403. Server-internal store calls (seed, attribution writes, the 30-day auto-expiry sweep) never pass through the routes, so they're untouched. - Seed the host page restricted BEFORE the DB is linked (no window). - Sanitize admin pricing overrides to finite, non-negative numbers so a bad PUT can't snapshot NaN/negative cost. - Add tests: managed page-route lockout (owner/admin 403, viewer/guest 404) + sweep/attribution stay ungated; host restricted-from-creation; pricing sanitization; provider SSE usage parsers (Anthropic/OpenAI-compat/ Llama). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X78XTAemKrFw1uJpDpCEaG
…ent-body page routes Two remaining HTTP paths bypassed the C1 managed-usage-DB write guards: - POST /api/importSpace overwrite mode (importOverwriteTx) could rewrite a usage-row's attribution properties, detach/re-home a row, inject a forged row, or overwrite the host page / DB schema. Strip any bundle entry targeting the managed usage DB in importBundle (host page id, a page claiming or currently tagged with database_id === usageDbId, or the DB itself) before the writers — covers HTTP + local (LocalDataClient) callers, mirrors isManagedPage against current state, and leaves ordinary pages/internal seed+attribution+sweep intact. - PUT /api/pages/:id and POST /api/pages (upsert ON CONFLICT name+data) could rename/body-overwrite a managed host/row. Add rejectManagedPage after the access gate (existence-hiding 404 for non-readers, managed 403 for mutators). Tests: hostile import-overwrite bundle leaves the usage DB intact while an ordinary page still imports; PUT/POST onto host/row → 403 (owner/admin), 404 (viewer/guest). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01X78XTAemKrFw1uJpDpCEaG
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
There was no attribution or accounting for agentic AI spend — no record of who consumed how many tokens or what it cost. This is the accounting half of the Epic (server-only AI keys + admin usage attribution).
Board: C1 · AI usage attribution backend + admin-only usage database + editable pricing.
Solution
Server-side token + cost attribution for every model request, logged to a server-managed, admin-only OpenBook database (a real product database, ACL-gated — using B's auto-expiry for retention).
onUsage?(TokenUsage)onGenerateOptions— Anthropic (message_start+ cache + cumulativemessage_delta), OpenAI-compat (stream_options.include_usagetrailing chunk; Mlx inherits), Llama (tokenize), Mock (deterministic).DEFAULT_PRICING$/Mtok table (Claude Opus/Sonnet/Haiku/Fable + common OpenAI; local providers = 0) with an admin-editable override insettings.aiPricing; effective = override → default → null (unknown model → tokens logged, cost null);cost_usdsnapshotted at write time; override values sanitized (finite, non-negative).visibility='restricted'(owner/admin only), markedmanagedand locked against every HTTP mutation — the 5 database routes, all 7 generic page routes (patch/props/delete/move/visibility/acl±), andimportSpaceoverwrite. Server-internal writes (seed, attribution, the 30-day auto-expiry sweep) bypass the lock. Non-admins are existence-hidden (404); a would-be admin mutator gets 403.GET/PUT /api/ai/pricing+PUT /api/ai/usage/retention(allrequireInstanceAdmin);client.getAiPricing()/setAiPricing()/setAiUsageRetention().Key files:
packages/server/src/ai/usage.ts(new),ai/providers.ts,ai/agent.ts,ai/routes.ts,app.ts,server.ts,store.ts,packages/sdk/src/{ai,client,routes,database}.ts. New deps: none.Before / After
No visual change — this slice is backend + routes only; the admin viewer + pricing UI is the next slice (C2).
restricteddatabase; viewers/guests get 404requireInstanceAdminTest procedure
pnpm verify→ green.packages/server/src/aiUsage.test.ts(19 tests): attribution per request (agent/generate); unknown-model null cost; restricted read (viewer/guest 404, owner/admin read); managed write-lockout across the 5 DB routes + 7 page routes + hostile import-overwrite + content-body routes; admin pricing/retention gates + non-admin 403; server-principal-only (spoofed body id ignored); idempotent seed; provider SSE parser fixtures (Anthropic/OpenAI/Llama,onUsagefires once); auto-expiry sweep + attribution stay ungated.Operational notes
settings. Every workspace gains an admin-onlyrestricted"AI usage" page (harmless; hidden from non-admins; C2 presents/hides it). If you'd prefer it seed lazily on first AI use, that's a small follow-up.main(2f549b2).Verify: green — sdk 171 · ui 939 · server 639 unit + e2e (251 embedded / 40 MCP). Gates cleared: code ✅ · security ✅ (blocker + import residual closed; re-verified).
🤖 Generated with Claude Code
https://claude.ai/code/session_01X78XTAemKrFw1uJpDpCEaG