feat(addie): add dated evaluation pricing cohort - #7312
Conversation
bccf770 to
53ace87
Compare
53ace87 to
5ea401d
Compare
There was a problem hiding this comment.
Ladon verdict: Approve
Approve — Addie eval/pricing refactor, no blocking findings.
This PR consolidates Addie eval pricing into a dated-cohort module, rewires the fixed-trace budget, updates the router-eval harness, and adjusts the OpenAI billing-identity/cache-write mappings. All changes are within Addie/eval scope (server/src/addie/** and tests), off the protocol wire surface (static/schemas/source/**, docs/reference/**, dist/**), so no changeset is warranted and none is expected.
Checks performed by the reviewer and confirmed against the gate rules:
- Accounting math verified (conservative subset/additive reservation, exact integer-micro path), immutability, fail-closed settlement, and error-contract preservation.
- OpenAI(subset 0.25)/Google(unsupported) cache-write data changes match the actual provider adapter usage mappings.
- No changeset required — change is off the published protocol surface.
Decision-table walk: no critical/high findings (row 1 n/a); gated_paths is false (row 2 n/a); no (deleted) high-risk reasons (row 3 n/a); no medium findings at all, so rows 4/5/8 n/a; no prior decision (row 6 n/a); no no-auto-approve team match (row 7 n/a). high_risk is true only via (added)/(modified) reasons with zero medium-or-higher findings — per the high_risk guidance this is not escalation-worthy on the flag alone. Falls through to row 9 → approve.
Non-blocking follow-up (informational): the Google fixed-trace pricing policy has a post-2026 wall-clock expiry worth revisiting before then.
Summary
Adds a credential-free, immutable, dated prospective pricing cohort for the current Addie evaluation candidates. The resolver returns immutable profiles and a domain-separated SHA-256 digest only after candidate, interval, evidence, and returned-model checks pass. It is accounting input only and does not create providers or authorize dispatch.
This revision makes the router evaluator consume that same immutable profile for admission, reservation, settlement, per-cell actual cost, and evidence hashing. Cache accounting is profile-driven: subset cache reads are never charged twice, while additive cache reads/writes are reserved conservatively and settled explicitly. Missing, unavailable, ambiguous, malformed, overlapping, or drifted cache usage fails closed; pre-dispatch refusal leaves no reservation.
OpenAI Luna cache writes are now recorded as a subset replacement bucket at
$0.25/MTok(the official model card's documented 1.25×$0.20uncached-input rate). Shared reservation accounting selects the costliest valid subset category, so that higher cache-write rate is covered before dispatch. Router reservations also retain the module-issued profile identity, digest, candidate, profile ID, provider, and model; settlement rejects cross-profile or frozen-substitute pricing before it can account spend.Router reservation handles are opaque frozen identity tokens. Their USD amount, active/dispatched lifecycle, and profile binding reside only in the owning ledger's private
WeakMap, preventing a caller from altering a refund or clearing another live reservation. Clone, proxy, lookalike, cross-ledger, inactive, and reused handles fail before accounting.The fixed-trace diagnostic fixtures now obtain their dated cohort evidence and rates from the canonical profile, rather than carrying stale standalone policy data.
Base and head
main/origin/mainatebe3f6c87bc61b2cab5bccd97253f845e056ceba(#7313); verified as this branch's exact merge base.5ea401d334829fecd3051e2d4d6ee8cda7dc044aThe #7305 dormant fixed-trace CLI, its local-
tsxchild contract, unknown-exposure evidence, and restored active coverage are retained unchanged. The fixed-trace budget continues to use the shared cache-aware dated pricing reservation helper.The #7313 fixed-trace smoke-overlay source and tests remain untouched by this branch and pass alongside the #7305 and pricing/router coverage.
Coverage and validation
2/4passed on the immediately preceding fix(addie): make fixed-trace dispatch dormant #7305 base rebase: 144 files, 1,979 tests.npm run typecheckpassed.git diff --check origin/main...HEADpassed.The repository's broad automatic pre-commit suite still encountered unrelated existing server database-fixture failures outside this change; the affected shard and focused changed-area suite above are green.
Review feedback disposition
5124430581.effectiveBeforeof2027-01-01T00:00:00.000Zwith the provider-owned Google pricing evidence. The focused expiry-boundary test proves selection becomes unavailable at that instant, and the router harness checks cohort availability before credentials or provider construction. Therefore expiry fails closed before dispatch. Operationally, the recorded Google rate/evidence must be refreshed before that date.Scope and authorization
No production provider activation, routing, replay execution behavior, rollout decision, quality claim, or spend authority is added. This remains a diagnostic-only pricing/admission change. No provider calls were made, no API key or secret is committed, and this PR is ready for merge with exact-head Sol and Ladon approval.