perf(dashboard): aggregate stats in SQL with TTL cache and cost backfill - #70
Merged
Conversation
GET /dashboard/api/stats was O(all retained rows) per page load: every non-runtime timeframe loaded the whole calls window three times (stats, logins, provider-keys), folded it in Python twice, re-priced unstamped rows per row per read, and re-parsed the price file per call — 20+ seconds at production volume. - host_store: usage_aggregate (one GROUPING SETS scan over idx_calls_ts for totals + by_caller/provider/model_family/route/served_model/ status/day), usage_rows_page (LIMIT'd recent), usage_count, usage_provider_stats, usage_connections, usage_event_ids_present, and an idempotent batched backfill_call_costs that stamps cost_usd/cost_basis='computed' on unpriced rows (unpriceable rows stay NULL and sum as 0, as before). - auth_proxy: the persistent-timeframe stats snapshot, provider credentials and login connections derive from the SQL rollups behind a 12s TTL cache keyed by the full query tuple (DASHBOARD_STATS_TTL_S); only not-yet-landed runtime rows are folded on top. _price_table() is memoized on file mtime. Cost backfill runs once at startup off the event loop. The dead double aggregation and _dedup_usage_rows are removed; the runtime path, /v1/usage and response shapes are unchanged. - tests: parity suite asserting the SQL path reproduces the reference Python aggregation exactly (totals, every group-by, daily buckets, recent page, filters, empty shapes), plus TTL-cache and backfill idempotency coverage. ~7x faster cold at 500k rows (0.85s vs 5.9s per full-window fold, of which the old path ran three per page load); repeat loads within the TTL do zero window scans.
|
Warning Review limit reached
Next review available in: 27 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
MuncleUscles
added a commit
that referenced
this pull request
Aug 6, 2026
The repo had no pre-merge checks. notify-ci.yml fires on push to main — after the merge button — and only dispatches to the private CI repo; PR checks were CodeRabbit alone, and branch protection required a review but zero status checks. tests/test_antseed_node.py was consequently red for a month across #63/#68/#69/#70/#71 and every one merged. Three jobs: - tests: pytest against a UTF8 Postgres service (SQL_ASCII makes psycopg return TEXT as bytes, which silently voids settings overrides), submodules checked out so the engine-backed tests are real, node present for the sidecar suite. - core-tests: the Lua policy core's unit + golden conformance vectors. - images: build both images, then smoke the artifact. The image smoke is the one that would have caught #95. A green suite proves the REPO is consistent and says nothing about what COPY put in the image — which is exactly how a control.js requiring ./ids.js shipped without it, died at import, and took every wallet endpoint down with a 502. scripts/check_sidecar_modules.js resolves (never executes) each shipped module's local imports inside the built image; verified to reproduce that failure against the old COPY list.
jmlago
pushed a commit
that referenced
this pull request
Aug 7, 2026
…#96) * ci: gate merges on tests, core conformance and a built-image smoke The repo had no pre-merge checks. notify-ci.yml fires on push to main — after the merge button — and only dispatches to the private CI repo; PR checks were CodeRabbit alone, and branch protection required a review but zero status checks. tests/test_antseed_node.py was consequently red for a month across #63/#68/#69/#70/#71 and every one merged. Three jobs: - tests: pytest against a UTF8 Postgres service (SQL_ASCII makes psycopg return TEXT as bytes, which silently voids settings overrides), submodules checked out so the engine-backed tests are real, node present for the sidecar suite. - core-tests: the Lua policy core's unit + golden conformance vectors. - images: build both images, then smoke the artifact. The image smoke is the one that would have caught #95. A green suite proves the REPO is consistent and says nothing about what COPY put in the image — which is exactly how a control.js requiring ./ids.js shipped without it, died at import, and took every wallet endpoint down with a 502. scripts/check_sidecar_modules.js resolves (never executes) each shipped module's local imports inside the built image; verified to reproduce that failure against the old COPY list. * ci: give the image smoke a database and fail fast on a dead container The router opens a host-store pool at startup, so booting it without Postgres proved only that it can fail to connect. Adds a postgres service to the images job and runs the container with --network host so it can reach it (a bridged container cannot see the runner's localhost). Also drops the '|| true' after docker run: a container that fails to start must fail the job immediately, not fall through to a curl loop that reports the same thing sixty seconds later.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The analytics dashboard took 20+ seconds to load at current request volume. Three panels (stats, login connections, provider-keys) each loaded the entire 30-day retention window from
callsinto Python dicts and aggregated there — no LIMIT, no GROUP BY — with the full set folded ~4× per page (filtered + unfiltered aggregation, daily totals, per-snapshot), the price file re-read and regex-parsed on every fold, and the 15s frontend auto-refresh re-running all of it from scratch.Fix
GROUPING SETSscan (host_store.usage_aggregate) computes totals + by_caller/provider/model_family/route/served_model/status/day; recent activity is aLIMIT 200page (usage_rows_page); login-connections and provider-keys panels get their own SQL rollups. All range-scanidx_calls_ts.cost_usd(cost_basis='computed') on unpriced rows, soSUM(cost_usd)is authoritative. Provider-reported costs untouched. Historical costs now freeze at backfill-time prices instead of being re-priced per read._price_table()memoized on file mtime.Response shape unchanged — no frontend changes.
timeframe=runtime,/v1/usage, and cost-accuracy paths untouched.Semantics preserved (test-pinned)
error =
status >= 400; NULL/'' bucket keys →unknown;tokens_totalfalls back to in+out; daily buckets are UTC calendar dates; rejects remain runtime-only (never persisted). One documented drift: a just-landed row still present in the runtime deque used to win latency dedup on the provider-keys panel; now the persisted row wins.Numbers
At ~500k retained rows: cold page load ~18s → ~0.85s of query work; warm (within TTL) loads touch only a ≤200-id probe for unlanded runtime rows.
Tests
New
tests/test_dashboard_stats_sql.py: exact parity between the SQL bundle and the old Python aggregation over 300 seeded rows × 6 filter combinations, provider/connections rollup parity, backfill idempotency + NULL scoping, TTL cache hit/miss, empty-DB shapes. Full suite: 470 passed, 2 skipped, 1 pre-existing unrelated failure (test_antseed_node.pyNode harness).