Skip to content

Data Contract

Jump to bottom
tavlean edited this page Jul 2, 2026 · 2 revisions

Data Contract — /api/export

How the extension gets its data, end to end. The producer-side twin of this page is docs/api-export.md in the site repo — when either side changes, update both.

The pipeline

  1. Tav edits models/benchmarks in the site's dev-only admin (~/Development/Tavlean/RankedAGI, data lives as committed JSON in data/); every write passes through the site's recompute choke point (recomputeAndWriteRagiModels()), which keeps ragi_* composites fresh.
  2. On the next Cloudflare Pages deploy, SvelteKit prerenders /api/export into a static JSON file (route: src/routes/api/export/+server.js; composition: src/lib/server/exportDataset.js, unit-tested). There is no live server — data freshness = deploy cadence. generatedAt is the build timestamp.
  3. The extension fetches it: view commands via useFetch (stale-while-revalidate — instant reopen from cache, background refresh, cache-control: public, max-age=3600); AI tools via loadDataset() (plain fetch + Raycast Cache, 1 h TTL, stale fallback when offline).
  4. Everything else — search, filtering, ranking, formatting — happens client-side in the extension. One payload, zero further requests.

Shape (v1)

{
  version: 1,
  generatedAt: ISO timestamp (build time),
  defaultSortKey: e.g. "ragi_code" (the site's default sort),
  benchmarks: BenchmarkDef[],   // non-archived defs
  models: ModelRow[]            // homepage-collapsed rows
}
  • models mirrors the site homepage exactly: one row per model family, chosen by the site's own representative logic ($lib/families.js — the endpoint imports it, so the extension can never disagree with the site). Collapsed rows carry __familySlug (link target) and __levelLabel (when a non-base level leads).
  • levels: rows of a multi-level family carry the raw per-level rows (base + reasoning variants; product variants stay separate list rows). Reasoning-level rows with zero real benchmark values are excluded — the site's own publish rule (score-record decision, addendum 17). A family whose only levels are empty gets NO levels array at all. This is why freshly-created admin test levels never show up in Raycast.
  • ModelRow is flat: identity fields (Slug, Model, Org, Version, Released, License, InCost/OutCost USD per M tokens, links: [{title, url}]) plus benchmark scores keyed directly on the row by benchmark key, plus ragi_* composites.
  • BenchmarkDef: key, name, shortName, subtitle, category, description, format (percent | integer | currency), ascending (true = lower is better), sortOrder, website, hidden, managedByRagi (true for the five RAGI composites), archived.

Rules that bite

  • Percent values are FRACTIONS 0–1 (0.9508 → "95.1%"). Multiply by 100 for display, one decimal.
  • Benchmark display name = name alone. It already contains any qualifier; name + subtitle doubles it. shortName exists for compact labels.
  • Missing score = key absent from the row. Render "—"; exclude from rankings.
  • hidden: true benchmarks don't appear in the site's default columns — the extension excludes them from lists/detail too (realBenchmarks filter).
  • Model page URL: https://rankedagi.com/models/<__familySlug ?? Slug>.

Evolution — the additive-only contract

The site may ADD fields at any time; it must never rename or remove one while version is 1. The extension must tolerate unknown fields (index signatures on the types). A breaking change ships as /api/export v2 alongside v1, and the extension migrates on its own schedule. This rule is written into the site's ROADMAP ("For future executors"), docs/api-export.md, and the endpoint's code comments — a site session that touches the endpoint will hit it three times.

Related endpoints (future features)

  • /api/score-provenance — slimmed per-score sources (drafts excluded), already live. Powers the "per-score sources" Later item in Roadmap.
  • /api/ragi-simulated — the ~3.7 MB estimate sidecar. Deliberately NOT consumed (v1 shows real scores only).

Local development

Until a deploy ships the endpoint (or when testing site-side changes): run the site dev server (npm run dev in the site repo), then set the extension's Data URL preference to http://127.0.0.1:<port>/api/export. A trimmed-but-real payload sample lives in this wiki as api-export-sample.json (2026-07-02) — code against it, not assumptions; regenerate with curl <dev-server>/api/export.

Clone this wiki locally