feat(billing): settle container usage charges

[pandemicsyn]

Summary

• Adds the first paid container-billing rollout for Gastown. Every five-minute heartbeat records accepted container time, converts only the newly accepted time into credits at the price captured when that container started, and then returns continue, warn, or stop based on the wallet balance after the debit.
• The meter performs the debit inside the same PostgreSQL transaction that records the heartbeat. Retried heartbeats reuse a stable identity and do not create another charge; the container interval lock and its settled-time watermark are the primary exactly-once guard.
• Adds compute_usage_charge, an immutable financial ledger for metered infrastructure. A charge records the source that produced it, the payer, SKU, quantity, applied rate, final microdollar amount, and the source timestamp. It is intentionally generic enough for future Worker billing without making container settlement state generic.
• The ledger is partitioned by its immutable created_at month. The migration creates the initial partitions, and the existing partition-maintenance cron keeps the current and next two months ready. This avoids an ever-growing single ledger table while ensuring a retry is routed to the same monthly partition as the original source event.
• Container intervals keep their own settled_billable_seconds watermark because cumulative settlement is container-specific. Container ledger rows use container_usage_segment plus the segment idempotency key as their source identity; future compute producers can use their own stable source identities and quantities.
• Integrates compute charges into personal and organization balance recomputation. The personal or aggregate organization wallet is debited atomically; organization member usage and Kilo Pass accounting remain intentionally unchanged.
• Enables the approved personal Gastown canary and stop enforcement. User and organization payer types have independent allowlists: the current production configuration enables only the named user, while all other users and all organizations remain shadow-only. A stop verdict blocks new work, allows a 120-second save/drain period, then force-stops the container and suppresses automatic restart.

sequenceDiagram
    participant G as Gastown container
    participant M as Usage meter
    participant I as Container interval
    participant L as compute_usage_charge
    participant W as Kilo-credit wallet

    Note over G,W: Paid session runs for 7 minutes
    G->>M: recordStart at 00:00
    M->>I: Create paid interval and snapshot payer plus SKU rate
    M-->>G: Accepted

    G->>M: Heartbeat at 05:00 for 300 accepted seconds
    M->>I: Lock interval and persist segment
    M->>I: Advance settled watermark to 300 seconds
    M->>L: Insert charge 1 with heartbeat source key and rate snapshot
    M->>W: Atomically debit charge 1
    M-->>G: Return continue warn or stop from post-debit balance

    G->>M: recordStop at 07:00 for 120 final seconds
    M->>I: Lock interval and persist final segment
    M->>I: Advance watermark to 420 seconds and close interval
    M->>L: Insert charge 2 with final segment source key
    M->>W: Atomically debit charge 2
    M-->>G: Return final post-debit verdict

Loading

Verification

Visual Changes

N/A

Reviewer Notes

• Billing is fail-closed: an invalid service list, malformed payer list, no eligible payer, or invalid warning threshold leaves usage in shadow mode. An empty user or organization list disables only that payer type.
• The production configuration targets gastown and user daef8451-f3f3-490e-93f1-21fafc2b005e; organization billing is disabled by an empty organization allowlist.
• The ledger has real restrictive payer/SKU foreign keys, but deliberately has no polymorphic telemetry foreign key or delete cascade. Source telemetry can be retained or deleted independently without deleting financial charges.
• A rate is snapshotted on paid intervals. Historical and recovery-created intervals remain shadow-only, so changing allowlists or prices does not retroactively change existing billing behavior.
• The existing Exa-specific partition helper, test, API route, Vercel cron path, and logging are renamed to the agnostic usage-partition maintenance surface. The same monthly scheduler now provisions both Exa and compute partitions; this PR does not add a second cron.

[pandemicsyn]

fyi - Just turning this on for me in prod for now. Will extend to our org after.

[kilo-code-bot]

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Executive Summary

Incremental review of cf23ae434 confirms the paid recordStart admission path now locks the open interval before the payer wallet row, matching heartbeat/stop settlement ordering, and the new PostgreSQL concurrency test proves the payer row stays lockable while supersede admission waits on the interval lock; the previously reported lock-order deadlock risk is resolved.

Files Reviewed (2 files)

• services/container-usage-meter/src/postgres.ts
• services/container-usage-meter/test/postgres.test.ts

Previous Review Summary (commit 6f41b15)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit 6f41b15)

Status: 1 Issue Found | Recommendation: Address before merge

Executive Summary

The paid recordStart admission path locks the payer wallet before the superseded interval, inverting the interval → payer lock order used by settlement and risking a Postgres deadlock under concurrent start/heartbeat traffic.

Overview

Severity	Count
CRITICAL	0
WARNING	1
SUGGESTION	0

Issue Details (click to expand)

WARNING

File	Line	Issue
services/container-usage-meter/src/postgres.ts	398	Payer-row lock taken before superseded-interval lock, inverting settlement's interval → payer lock order; concurrent paid start + heartbeat on the same payer can deadlock (40P01)

Files Reviewed (28 files)

• .specs/gastown-usage-based-billing.md
• apps/web/src/app/api/cron/usage-partition-maintenance/route.ts (renamed)
• apps/web/src/lib/usage-partitions.ts / usage-partitions.test.ts (renamed, compute partition provisioner added)
• apps/web/src/lib/user/recompute-balances.ts
• apps/web/src/lib/recomputeOrganizationBalances.ts
• apps/web/src/lib/ai-gateway/spend-writer-audit.test.ts
• apps/web/src/routers/admin/cloud-billing-skus-router.test.ts
• apps/web/src/scripts/db/exa-usage-log-indexes.ts
• apps/web/src/tests/setup/workerSetup.ts
• apps/web/vercel.json
• packages/container-usage/src/contracts.ts
• packages/db/src/schema.ts
• packages/db/src/migrations/0206_supreme_sally_floyd.sql
• services/container-usage-meter/src/billing-config.ts / billing-config.test.ts - fail-closed allowlist parsing verified
• services/container-usage-meter/src/meter.ts / meter.test.ts
• services/container-usage-meter/src/postgres.ts - 1 issue (lock ordering)
• services/container-usage-meter/test/postgres.test.ts - paid personal/org settlement coverage verified against balance math
• services/container-usage-meter/wrangler.jsonc / worker-configuration.d.ts
• services/gastown/src/billing/container-usage-state.billing.ts
• services/gastown/src/dos/TownContainer.do.ts - stop-verdict phase transition, restart guard, and TownDO enforcement loop verified
• services/gastown/container/src/process-manager.ts
• services/gastown/scripts/patch-worker-types.mjs
• services/gastown/wrangler.jsonc / worker-configuration.d.ts

Generated migration snapshot/journal files were excluded from review per repo rules. Settlement math (cumulative floor watermark), idempotent retry handling, fail-closed billing config, partition provisioning, and balance recompute integration all verified correct.

Fix these issues in Kilo Cloud

---

_{Reviewed by kimi-k3 · Input: 76.2K · Output: 15.4K · Cached: 877.6K}

_{Review guidance: REVIEW.md from base branch main}