diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index c64c018..1f74d00 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -16,6 +16,7 @@ jobs:
           node-version: 22
           cache: pnpm
       - run: pnpm install --frozen-lockfile
+      # evals imports extractkit's built types, so build first
+      - run: pnpm build
       - run: pnpm typecheck
       - run: pnpm test
-      - run: pnpm build
diff --git a/.gitignore b/.gitignore
index 85baafb..f469d56 100644
--- a/.gitignore
+++ b/.gitignore
@@ -7,3 +7,4 @@ dist/
 *.log
 .turbo/
 coverage/
+.data/
diff --git a/CLAUDE.md b/CLAUDE.md
index 4f2cf3a..fd103e7 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -4,7 +4,7 @@ extractkit is an open-source TypeScript document-extraction engine: Zod schema +
 
 ## Current Phase
 
-**Phase 1 (core library) shipped; Phase 2 (evals) is next.** `packages/core` is implemented and tested against mock models — it has not been validated against live providers yet; that happens as Phase 2 stands up. `packages/evals` and `apps/playground` do not exist yet. Keep README/ROADMAP/docs in sync with what actually ships. Requesting the DocILE dataset token (see ROADMAP Phase 0) is a pending human action that gates the Phase 2 data pull.
+**Phase 2 (evals) harness shipped; live runs and the DocILE half are pending.** `packages/core` and the `packages/evals` harness are implemented and tested against mock models. The 25 CORD-v2 receipts are pinned in `packages/evals/data/manifest.json`; the 25 DocILE invoices are blocked on the dataset token (ROADMAP Phase 0, pending human action) — loader and curation script are ready. The eval lineup spans Anthropic, OpenAI, and Google Gemini (`packages/evals/src/models.ts`); a run includes every provider whose API key is set, or the subset named in `EVAL_PROVIDERS`. The first live eval run (needs at least one provider key — `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, or `GOOGLE_GENERATIVE_AI_API_KEY`) doubles as core's first live-provider validation and fills the benchmark page + README table via `pnpm report`. `apps/playground` does not exist yet. Keep README/ROADMAP/docs in sync with what actually ships.
 
 ## Planned Architecture
 
diff --git a/README.md b/README.md
index c487fd7..1712404 100644
--- a/README.md
+++ b/README.md
@@ -11,11 +11,19 @@ TypeScript has structured-output libraries (instructor-js, AI SDK `generateObjec
 ## v1
 
 - **Core library** (`packages/core`, shipped) — Zod schema + PDF/image → validated JSON with per-field `{ value, confidence, page, bbox }`. Provider-agnostic via the Vercel AI SDK. Document validation, typed failure handling, repair retries, streaming, and cost tracking built in. [Usage docs →](./packages/core/README.md)
-- **Eval harness** (next) — public benchmark on ~50 real documents (invoices/receipts): field accuracy per model, grounding accuracy, cost per 1k docs.
+- **Eval harness** (`packages/evals`, harness shipped) — public benchmark on ~50 pinned real documents (CORD-v2 receipts + DocILE invoices): field accuracy per model, grounding accuracy, cost per 1k docs. Fully reproducible — documents pinned by checksum, reports generated only from recorded runs. [Reproduce it →](./packages/evals/README.md)
 - **Playground** (planned) — drag-drop a document, watch fields extract; hover a JSON field to highlight its source region on the page.
 
 See [ROADMAP.md](./ROADMAP.md) for the build plan.
 
+## Benchmark
+
+<!-- benchmark:start -->
+
+*No results published yet — the first live eval run is pending. Numbers will appear here only from recorded runs; see [`packages/evals`](./packages/evals) to reproduce.*
+
+<!-- benchmark:end -->
+
 ## Scope
 
 General-purpose business documents: invoices, receipts, contracts.
diff --git a/ROADMAP.md b/ROADMAP.md
index c3d83b5..59fa914 100644
--- a/ROADMAP.md
+++ b/ROADMAP.md
@@ -16,9 +16,11 @@ Tested against mock models only so far; first live-provider validation happens w
 
 ## Phase 2 — Evals (`packages/evals`)
 
-- [ ] Harness: accuracy per field per model, grounding accuracy (predicted vs. ground-truth bbox), cost per 1k docs
-- [ ] Benchmark page generated from real runs (engineering target: >90% field accuracy on the invoice set)
-- [ ] Benchmark table in README
+- [x] Harness: accuracy per field per model, grounding accuracy (predicted vs. ground-truth bbox), cost per 1k docs — tested against mock models; report generation from recorded runs
+- [x] Receipt half pinned: 25 CORD-v2 test docs curated by mapping-consistency checks, pinned by id + SHA-256 in `packages/evals/data/manifest.json`
+- [ ] Invoice half pinned: blocked on the DocILE token (Phase 0 human action); curation script is ready
+- [ ] First live eval run (needs a provider key — `ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, and/or `GOOGLE_GENERATIVE_AI_API_KEY`) → benchmark page generated from real runs (engineering target: >90% field accuracy on the invoice set)
+- [ ] Benchmark table in README (markers in place; filled by `pnpm report` from a recorded run)
 
 ## Phase 3 — Playground (`apps/playground`)
 
diff --git a/packages/core/README.md b/packages/core/README.md
index 7a5613a..8cfef08 100644
--- a/packages/core/README.md
+++ b/packages/core/README.md
@@ -10,7 +10,7 @@
 npm install extractkit ai zod
 ```
 
-`ai` (Vercel AI SDK v7) and `zod` (v4) are peer dependencies. Bring any AI SDK provider, e.g. `@ai-sdk/anthropic`.
+`ai` (Vercel AI SDK v7) and `zod` (v4) are peer dependencies. Bring any AI SDK provider — e.g. `@ai-sdk/anthropic`, `@ai-sdk/openai`, or `@ai-sdk/google` — and pass its model to `extract`.
 
 ## Quickstart
 
diff --git a/packages/evals/README.md b/packages/evals/README.md
new file mode 100644
index 0000000..d80fbb0
--- /dev/null
+++ b/packages/evals/README.md
@@ -0,0 +1,43 @@
+# @extractkit/evals
+
+The extractkit benchmark harness: per-field value accuracy, grounding accuracy (predicted vs. annotated bounding box), and cost per 1k documents, measured on ~50 pinned public documents — 25 receipts from [CORD-v2](https://huggingface.co/datasets/naver-clova-ix/cord-v2) (NAVER CLOVA, CC BY 4.0) and 25 invoices from [DocILE](https://docile.rossum.ai/) (Rossum). Dataset rationale: [docs/benchmark-dataset.md](../../docs/benchmark-dataset.md).
+
+Documents are never vendored. [`data/manifest.json`](./data/manifest.json) pins each document by id + SHA-256, and the fetch script pulls them from their canonical hosts and verifies every checksum, so every published number is reproducible on exactly the same bytes.
+
+## Reproducing the benchmark
+
+```sh
+pnpm install && pnpm build          # from the repo root; evals imports core's build
+
+cd packages/evals
+pnpm fetch-data                     # CORD parquet (~230 MB) into .data/, checksum-verified
+                                    # set DOCILE_TOKEN to also fetch the DocILE invoice half
+export ANTHROPIC_API_KEY=...        # and/or OPENAI_API_KEY / GOOGLE_GENERATIVE_AI_API_KEY
+pnpm run-eval                       # extract every pinned doc with every model → results/run-<ts>.json
+pnpm report [results/run-....json]  # regenerate docs/benchmark.md + the README table
+```
+
+Without `DOCILE_TOKEN` the receipt half still runs; the invoice half needs a free token from [docile.rossum.ai](https://docile.rossum.ai/) (DocILE terms prohibit redistributing the documents, so every runner requests their own).
+
+### Choosing providers
+
+The lineup (`src/models.ts`) spans three providers — **Anthropic**, **OpenAI**, and **Google Gemini** — with three vision-capable tiers each and their public list pricing:
+
+| Provider | Key | Models |
+| --- | --- | --- |
+| Anthropic | `ANTHROPIC_API_KEY` | `claude-opus-4-8`, `claude-sonnet-5`, `claude-haiku-4-5` |
+| OpenAI | `OPENAI_API_KEY` | `gpt-5.6-sol`, `gpt-5.6-luna`, `gpt-5.4-mini` |
+| Google | `GOOGLE_GENERATIVE_AI_API_KEY` | `gemini-2.5-pro`, `gemini-2.5-flash`, `gemini-2.5-flash-lite` |
+
+By default the run includes every provider whose API key is set, so exporting only `OPENAI_API_KEY` benchmarks OpenAI alone — a cheaper way to iterate than the full Anthropic lineup. Pin the selection explicitly with `EVAL_PROVIDERS` (comma-separated, e.g. `EVAL_PROVIDERS=openai,google`); each named provider must have its key set.
+
+`pnpm pin` re-runs curation and rewrites the manifest — only needed when changing the selection, not to reproduce a run.
+
+## How scoring works
+
+- **Ground truth mapping** (`src/datasets/`): each dataset's native labels are mapped onto the demo Zod schemas in `src/schemas.ts` (receipt ← CORD `gt_parse` + `valid_line`; invoice ← DocILE KILE/LIR fieldtypes). Documents whose annotations can't be mapped confidently are rejected at pin time (`CordMappingError` / `DocileMappingError`) rather than silently mis-scored; the mapping code is in-repo so it can be audited.
+- **Value accuracy**: normalized comparison per field kind — whitespace/case for text, digits-and-sign for amounts ("24,000" ≡ "24.000" ≡ "24000"), lenient numeric parse for quantities ("2.00" ≡ "2"). A field the document doesn't carry counts as correct only when the model returns null. Line items align by printed order; extra predicted items are reported separately as hallucinations.
+- **Grounding**: among fields with a correct value and an annotated region, best IoU between the predicted bbox and any acceptable ground-truth region on that page; hit@0.5 is the headline number, missing bbox or wrong page scores 0.
+- **Cost**: measured token usage priced at the per-MTok list prices in `src/models.ts`.
+
+Raw per-field results for every run are serialized to `results/`, and reports are generated only from those records — no hand-entered numbers.
diff --git a/packages/evals/data/manifest.json b/packages/evals/data/manifest.json
new file mode 100644
index 0000000..237db61
--- /dev/null
+++ b/packages/evals/data/manifest.json
@@ -0,0 +1,142 @@
+{
+  "version": 1,
+  "cord": {
+    "dataset": "naver-clova-ix/cord-v2",
+    "revision": "7f0115a4b758a71d6473b8d085751692da2fef98",
+    "file": "data/test-00000-of-00001-9c204eb3f4e11791.parquet",
+    "sha256": "51c65f1788faff392abe2a0b55b023eb23e9be551c509138eaa3a832514224e7",
+    "split": "test",
+    "docs": [
+      {
+        "rowIndex": 0,
+        "imageId": 0,
+        "imageSha256": "8612d04b70f430f3aef07fbbd5200e382dcc4152b344cc2eff9f735f05a257c8"
+      },
+      {
+        "rowIndex": 1,
+        "imageId": 1,
+        "imageSha256": "5852cbe48df03646524e717e987eb3d0ac329a52fd192070b5fda1222a053789"
+      },
+      {
+        "rowIndex": 2,
+        "imageId": 2,
+        "imageSha256": "571ac6af26e75de8c06747a8c536c778f19a4562879090813571eddc74603d34"
+      },
+      {
+        "rowIndex": 3,
+        "imageId": 3,
+        "imageSha256": "8f73f0c4803ea8815040a50e3389d4f3fa15e8c28cb6d50ea9854f6f6c22bd63"
+      },
+      {
+        "rowIndex": 4,
+        "imageId": 4,
+        "imageSha256": "8f3eee7068c96e86cdb2e4b5c53085cb5e1439462edd55c373548cb1962801ad"
+      },
+      {
+        "rowIndex": 5,
+        "imageId": 5,
+        "imageSha256": "0b63b666b54ea615d1d05733ee2c6fc24f20e1a6c77b333872fa80fff28b2c48"
+      },
+      {
+        "rowIndex": 6,
+        "imageId": 6,
+        "imageSha256": "1c091eb7436b3e5a813eb6492c0350ef4eb14e3fdad78d42da35df669c757292"
+      },
+      {
+        "rowIndex": 7,
+        "imageId": 7,
+        "imageSha256": "810d3545f4105b19757c4b8b35934a477cde8cd1875302d9092315d2f960ef93"
+      },
+      {
+        "rowIndex": 8,
+        "imageId": 8,
+        "imageSha256": "58716693a6c0733358e66e64d7064a5a845424280bb55f9b5fa137b1af8c92fe"
+      },
+      {
+        "rowIndex": 9,
+        "imageId": 9,
+        "imageSha256": "0411c4d9beb9be65165fdb70c2b60f8bbf9035cc9661be2b8d4c73bbfcff7972"
+      },
+      {
+        "rowIndex": 10,
+        "imageId": 10,
+        "imageSha256": "2a5d18c199bdbb75010a54430429cf694b35bdca8bf560d1d0db1fbb090dedb8"
+      },
+      {
+        "rowIndex": 11,
+        "imageId": 11,
+        "imageSha256": "d92479ab212713e7a65e56aacc5e68c0713597476608cbc394d997971043010b"
+      },
+      {
+        "rowIndex": 12,
+        "imageId": 12,
+        "imageSha256": "b5f8a3a9a5431b8e38a8e5a16011d6a5a457f7ae4c4ef31da77e589c0b1ad882"
+      },
+      {
+        "rowIndex": 14,
+        "imageId": 14,
+        "imageSha256": "9dcf1f9f86c909b762767cb07d1cc0db30773d556154ce42e0210f65270ee39d"
+      },
+      {
+        "rowIndex": 15,
+        "imageId": 15,
+        "imageSha256": "f1cd67a7d67d04ba8d5b685553b1b5addbf2039d5954ab08c92139c0ad88a796"
+      },
+      {
+        "rowIndex": 16,
+        "imageId": 16,
+        "imageSha256": "eef326565b3945217a69afd84c893afdffd03a2f487943661c2423e6d6ec0d7d"
+      },
+      {
+        "rowIndex": 17,
+        "imageId": 17,
+        "imageSha256": "4322ab5ec4f0b1365e08af0c31288e65e8cbd2c8eb5241adfb329244acbbb30b"
+      },
+      {
+        "rowIndex": 18,
+        "imageId": 18,
+        "imageSha256": "88c35bbd2e2a634e89f2c5bfd30f7b530b1082c1f4e6e7cb3cb913f0b46c3868"
+      },
+      {
+        "rowIndex": 19,
+        "imageId": 19,
+        "imageSha256": "b076921ed83bdf900cab539f219cac8dddfaa4a99c13f7d66d7303d9f8c4c8c7"
+      },
+      {
+        "rowIndex": 20,
+        "imageId": 20,
+        "imageSha256": "6213439292eb4c2ad7e04afab6bb5bc7d1d5925d18dc392c36f233d394ca7c19"
+      },
+      {
+        "rowIndex": 29,
+        "imageId": 29,
+        "imageSha256": "1f065c268473d727282005e57441c55c3249afe0da54f3bc08823d0cbe730610"
+      },
+      {
+        "rowIndex": 30,
+        "imageId": 30,
+        "imageSha256": "52879816c28ebc3a7bc777c31685fb71f36bc4d1f7d7dc9c0b743fc5aced84de"
+      },
+      {
+        "rowIndex": 35,
+        "imageId": 35,
+        "imageSha256": "453a4ad5aadf63b2a7efdfa0915d45fd72c426254ab8f7c84d5621d56445794c"
+      },
+      {
+        "rowIndex": 38,
+        "imageId": 38,
+        "imageSha256": "b7afc72ae79e1c0b23b46c83ae89031bb08ce5aec5e532e7a6f74b9511dc7fda"
+      },
+      {
+        "rowIndex": 41,
+        "imageId": 41,
+        "imageSha256": "994b983a7f51670ca67b35bbaf9a11c8852abf1711651448d4ce39f8f8119d6f"
+      }
+    ]
+  },
+  "docile": {
+    "archive": "annotated-trainval",
+    "split": "val",
+    "docs": []
+  }
+}
diff --git a/packages/evals/package.json b/packages/evals/package.json
new file mode 100644
index 0000000..ef1db67
--- /dev/null
+++ b/packages/evals/package.json
@@ -0,0 +1,33 @@
+{
+  "name": "@extractkit/evals",
+  "private": true,
+  "version": "0.0.0",
+  "description": "Benchmark harness for extractkit: per-field value accuracy, grounding accuracy, and cost per 1k docs on pinned public invoices and receipts.",
+  "type": "module",
+  "engines": {
+    "node": ">=20.19"
+  },
+  "scripts": {
+    "fetch-data": "tsx scripts/fetch.ts",
+    "pin": "tsx scripts/pin.ts",
+    "run-eval": "tsx scripts/run.ts",
+    "report": "tsx scripts/report.ts",
+    "test": "vitest run",
+    "typecheck": "tsc --noEmit"
+  },
+  "dependencies": {
+    "@ai-sdk/anthropic": "^4.0.10",
+    "@ai-sdk/google": "^4.0.12",
+    "@ai-sdk/openai": "^4.0.11",
+    "ai": "^7.0.16",
+    "extractkit": "workspace:*",
+    "hyparquet": "^1.14.0",
+    "zod": "^4.4.3"
+  },
+  "devDependencies": {
+    "@types/node": "^26.1.0",
+    "tsx": "^4.19.2",
+    "typescript": "^6.0.3",
+    "vitest": "^4.1.10"
+  }
+}
diff --git a/packages/evals/scripts/fetch.ts b/packages/evals/scripts/fetch.ts
new file mode 100644
index 0000000..29d68c1
--- /dev/null
+++ b/packages/evals/scripts/fetch.ts
@@ -0,0 +1,82 @@
+// Fetches the pinned benchmark data from its canonical hosts into .data/.
+// CORD-v2 (ungated) always; DocILE only when DOCILE_TOKEN is set.
+import { spawn } from 'node:child_process';
+import { createWriteStream } from 'node:fs';
+import { access, mkdir, readFile, rm } from 'node:fs/promises';
+import { Readable } from 'node:stream';
+import { pipeline } from 'node:stream/promises';
+import { loadManifest, sha256Hex } from '../src/manifest.js';
+import { CORD_PARQUET_PATH, DATA_DIR, DOCILE_ROOT } from './paths.js';
+
+async function exists(path: string): Promise<boolean> {
+  try {
+    await access(path);
+    return true;
+  } catch {
+    return false;
+  }
+}
+
+async function download(url: string, dest: string): Promise<void> {
+  const res = await fetch(url);
+  if (!res.ok || res.body === null) throw new Error(`GET ${url} failed: ${res.status} ${res.statusText}`);
+  await pipeline(Readable.fromWeb(res.body as import('node:stream/web').ReadableStream), createWriteStream(dest));
+}
+
+async function fetchCord(): Promise<void> {
+  const { manifest } = await loadManifest();
+  const { dataset, revision, file, sha256 } = manifest.cord;
+  if (await exists(CORD_PARQUET_PATH)) {
+    const digest = sha256Hex(await readFile(CORD_PARQUET_PATH));
+    if (digest === sha256) {
+      console.log(`cord: ${file} already cached and verified`);
+      return;
+    }
+    console.log('cord: cached parquet has wrong checksum, re-downloading');
+    await rm(CORD_PARQUET_PATH);
+  }
+  const url = `https://huggingface.co/datasets/${dataset}/resolve/${revision}/${file}`;
+  console.log(`cord: downloading ${url}`);
+  await download(url, CORD_PARQUET_PATH);
+  const digest = sha256Hex(await readFile(CORD_PARQUET_PATH));
+  if (digest !== sha256) {
+    await rm(CORD_PARQUET_PATH);
+    throw new Error(`cord: downloaded parquet sha256 ${digest} does not match pinned ${sha256}`);
+  }
+  console.log('cord: downloaded and verified');
+}
+
+async function fetchDocile(): Promise<void> {
+  const { manifest } = await loadManifest();
+  if (await exists(`${DOCILE_ROOT}/annotations`)) {
+    console.log('docile: dataset already present');
+    return;
+  }
+  const token = process.env['DOCILE_TOKEN'];
+  if (token === undefined) {
+    console.log(
+      'docile: skipped — set DOCILE_TOKEN to fetch the invoice half.\n' +
+        '  Request a free token at https://docile.rossum.ai/ (the CORD receipt half works without it).',
+    );
+    return;
+  }
+  const archive = manifest.docile.archive;
+  const zipPath = `${DOCILE_ROOT}.zip`;
+  // Download URL shape from rossumai/docile download_dataset.sh; the token is
+  // a path segment, so never log the URL.
+  console.log(`docile: downloading ${archive}.zip (several GB, this takes a while)`);
+  await download(`https://docile-dataset-rossum.s3.eu-west-1.amazonaws.com/${token}/${archive}.zip`, zipPath);
+  await mkdir(DOCILE_ROOT, { recursive: true });
+  console.log('docile: extracting');
+  await new Promise<void>((resolve, reject) => {
+    const child = spawn('unzip', ['-quo', zipPath, '-d', DOCILE_ROOT], { stdio: 'inherit' });
+    child.on('error', reject);
+    child.on('exit', (code) => (code === 0 ? resolve() : reject(new Error(`unzip exited with ${code}`))));
+  });
+  await rm(zipPath);
+  console.log('docile: done');
+}
+
+await mkdir(DATA_DIR, { recursive: true });
+await fetchCord();
+await fetchDocile();
diff --git a/packages/evals/scripts/paths.ts b/packages/evals/scripts/paths.ts
new file mode 100644
index 0000000..a8d3459
--- /dev/null
+++ b/packages/evals/scripts/paths.ts
@@ -0,0 +1,9 @@
+import { fileURLToPath } from 'node:url';
+
+/** Local cache for fetched benchmark data; gitignored, never vendored. */
+export const DATA_DIR = fileURLToPath(new URL('../.data', import.meta.url));
+export const CORD_PARQUET_PATH = fileURLToPath(new URL('../.data/cord-test.parquet', import.meta.url));
+export const DOCILE_ROOT = fileURLToPath(new URL('../.data/docile', import.meta.url));
+export const RESULTS_DIR = fileURLToPath(new URL('../results', import.meta.url));
+export const BENCHMARK_PAGE_PATH = fileURLToPath(new URL('../../../docs/benchmark.md', import.meta.url));
+export const README_PATH = fileURLToPath(new URL('../../../README.md', import.meta.url));
diff --git a/packages/evals/scripts/pin.ts b/packages/evals/scripts/pin.ts
new file mode 100644
index 0000000..8f72f84
--- /dev/null
+++ b/packages/evals/scripts/pin.ts
@@ -0,0 +1,151 @@
+// Curates the benchmark selection and writes data/manifest.json.
+// Deterministic: same source data in, same manifest out. Documents whose
+// ground truth cannot be mapped confidently are skipped, never mis-scored.
+import { access, mkdir, readFile, writeFile } from 'node:fs/promises';
+import { dirname, join } from 'node:path';
+import { cordToGroundTruth, CordMappingError } from '../src/datasets/cord.js';
+import { readCordRows } from '../src/datasets/cord-source.js';
+import { docileToGroundTruth, DocileMappingError } from '../src/datasets/docile.js';
+import { readDocileDoc, readDocileSplit } from '../src/datasets/docile-source.js';
+import { MANIFEST_PATH, sha256Hex, type CordPin, type DocilePin, type Manifest } from '../src/manifest.js';
+import { CORD_PARQUET_PATH, DOCILE_ROOT } from './paths.js';
+
+const DOCS_PER_HALF = 25;
+
+/** The pinned CORD-v2 source: main-branch test parquet at a fixed revision.
+ * sha256 is the file's LFS oid, cross-checked against the download. */
+const CORD_SOURCE = {
+  dataset: 'naver-clova-ix/cord-v2',
+  revision: '7f0115a4b758a71d6473b8d085751692da2fef98',
+  file: 'data/test-00000-of-00001-9c204eb3f4e11791.parquet',
+  sha256: '51c65f1788faff392abe2a0b55b023eb23e9be551c509138eaa3a832514224e7',
+  split: 'test',
+};
+
+const DOCILE_SOURCE = { archive: 'annotated-trainval', split: 'val' };
+
+async function exists(path: string): Promise<boolean> {
+  try {
+    await access(path);
+    return true;
+  } catch {
+    return false;
+  }
+}
+
+/**
+ * Deterministic diversity selection: bucket candidates by a key, then take
+ * round-robin from each bucket in candidate order until `count` are chosen.
+ */
+function roundRobin<T>(candidates: T[], bucketOf: (c: T) => string, count: number): T[] {
+  const buckets = new Map<string, T[]>();
+  for (const c of candidates) {
+    const key = bucketOf(c);
+    const list = buckets.get(key) ?? [];
+    list.push(c);
+    buckets.set(key, list);
+  }
+  const keys = [...buckets.keys()].sort();
+  const picked: T[] = [];
+  for (let round = 0; picked.length < count; round++) {
+    let took = false;
+    for (const key of keys) {
+      const bucket = buckets.get(key) as T[];
+      if (round < bucket.length && picked.length < count) {
+        picked.push(bucket[round] as T);
+        took = true;
+      }
+    }
+    if (!took) break;
+  }
+  return picked;
+}
+
+async function pinCord(): Promise<CordPin[]> {
+  if (!(await exists(CORD_PARQUET_PATH))) {
+    throw new Error(`cord: ${CORD_PARQUET_PATH} not found — run \`pnpm fetch-data\` first`);
+  }
+  const rows = await readCordRows(CORD_PARQUET_PATH);
+  const candidates: Array<CordPin & { lineItemCount: number }> = [];
+  for (const row of rows) {
+    try {
+      const { lineItemCount } = cordToGroundTruth(row.groundTruth);
+      candidates.push({
+        rowIndex: row.rowIndex,
+        imageId: row.groundTruth.meta.image_id,
+        imageSha256: sha256Hex(row.imageBytes),
+        lineItemCount,
+      });
+    } catch (err) {
+      if (!(err instanceof CordMappingError)) throw err;
+      console.log(`cord row ${row.rowIndex}: skipped (${err.message})`);
+    }
+  }
+  console.log(`cord: ${candidates.length}/${rows.length} rows map cleanly`);
+  if (candidates.length < DOCS_PER_HALF) throw new Error('cord: not enough cleanly-mapping rows to pin');
+  const picked = roundRobin(
+    candidates,
+    (c) => (c.lineItemCount === 1 ? 'single-item' : c.lineItemCount <= 3 ? 'few-items' : 'many-items'),
+    DOCS_PER_HALF,
+  ).sort((a, b) => a.rowIndex - b.rowIndex);
+  return picked.map(({ rowIndex, imageId, imageSha256 }) => ({ rowIndex, imageId, imageSha256 }));
+}
+
+async function pinDocile(previous: DocilePin[]): Promise<DocilePin[]> {
+  if (!(await exists(join(DOCILE_ROOT, `${DOCILE_SOURCE.split}.json`)))) {
+    console.log(
+      previous.length > 0
+        ? 'docile: dataset not present locally — keeping existing pins'
+        : 'docile: dataset not present locally — invoice half stays unpinned (needs DOCILE_TOKEN + `pnpm fetch-data`)',
+    );
+    return previous;
+  }
+  const docIds = (await readDocileSplit(DOCILE_ROOT, DOCILE_SOURCE.split)).sort();
+  const candidates: Array<DocilePin & { multiPage: boolean }> = [];
+  for (const docId of docIds) {
+    try {
+      const { pdfBytes, annotation } = await readDocileDoc(DOCILE_ROOT, docId);
+      if (annotation.metadata.document_type !== 'tax_invoice') continue;
+      const { fields, lineItemCount } = docileToGroundTruth(annotation);
+      const byPath = new Map(fields.map((f) => [f.path, f.value]));
+      const required = ['vendorName', 'invoiceNumber', 'issueDate', 'total'];
+      if (required.some((path) => byPath.get(path) == null)) continue;
+      if (lineItemCount === 0 || byPath.get('lineItems.0.description') == null) continue;
+      candidates.push({
+        docId,
+        pdfSha256: sha256Hex(pdfBytes),
+        annotationSha256: sha256Hex(await readFile(join(DOCILE_ROOT, 'annotations', `${docId}.json`))),
+        multiPage: annotation.metadata.page_count > 1,
+      });
+    } catch (err) {
+      if (!(err instanceof DocileMappingError)) throw err;
+      console.log(`docile ${docId}: skipped (${err.message})`);
+    }
+  }
+  console.log(`docile: ${candidates.length} candidate invoices in ${DOCILE_SOURCE.split}`);
+  if (candidates.length < DOCS_PER_HALF) throw new Error('docile: not enough candidate invoices to pin');
+  const picked = roundRobin(candidates, (c) => (c.multiPage ? 'multi-page' : 'single-page'), DOCS_PER_HALF).sort(
+    (a, b) => a.docId.localeCompare(b.docId),
+  );
+  return picked.map(({ docId, pdfSha256, annotationSha256 }) => ({ docId, pdfSha256, annotationSha256 }));
+}
+
+let previousDocile: DocilePin[] = [];
+try {
+  const existing = JSON.parse(await readFile(MANIFEST_PATH, 'utf8')) as Manifest;
+  previousDocile = existing.docile.docs;
+} catch {
+  // First pin run; no manifest yet.
+}
+
+const manifest: Manifest = {
+  version: 1,
+  cord: { ...CORD_SOURCE, docs: await pinCord() },
+  docile: { ...DOCILE_SOURCE, docs: await pinDocile(previousDocile) },
+};
+
+await mkdir(dirname(MANIFEST_PATH), { recursive: true });
+await writeFile(MANIFEST_PATH, `${JSON.stringify(manifest, null, 2)}\n`);
+console.log(
+  `pinned ${manifest.cord.docs.length} receipts + ${manifest.docile.docs.length} invoices → ${MANIFEST_PATH}`,
+);
diff --git a/packages/evals/scripts/report.ts b/packages/evals/scripts/report.ts
new file mode 100644
index 0000000..4b5a122
--- /dev/null
+++ b/packages/evals/scripts/report.ts
@@ -0,0 +1,36 @@
+// Regenerates docs/benchmark.md and the README benchmark table from a run
+// record. Usage: pnpm report [results/run-....json] (defaults to the latest).
+import { readdir, readFile, writeFile } from 'node:fs/promises';
+import { join } from 'node:path';
+import { renderBenchmarkPage, renderReadmeTable } from '../src/report.js';
+import type { RunRecord } from '../src/types.js';
+import { BENCHMARK_PAGE_PATH, README_PATH, RESULTS_DIR } from './paths.js';
+
+const README_START = '<!-- benchmark:start -->';
+const README_END = '<!-- benchmark:end -->';
+
+async function latestRunPath(): Promise<string> {
+  const files = (await readdir(RESULTS_DIR)).filter((f) => f.startsWith('run-') && f.endsWith('.json')).sort();
+  const last = files[files.length - 1];
+  if (last === undefined) throw new Error(`no run records in ${RESULTS_DIR} — run \`pnpm run-eval\` first`);
+  return join(RESULTS_DIR, last);
+}
+
+const runPath = process.argv[2] ?? (await latestRunPath());
+const record = JSON.parse(await readFile(runPath, 'utf8')) as RunRecord;
+
+await writeFile(BENCHMARK_PAGE_PATH, renderBenchmarkPage(record));
+console.log(`wrote ${BENCHMARK_PAGE_PATH}`);
+
+const readme = await readFile(README_PATH, 'utf8');
+const start = readme.indexOf(README_START);
+const end = readme.indexOf(README_END);
+if (start === -1 || end === -1) {
+  console.log(`README has no ${README_START} … ${README_END} markers; README table:\n`);
+  console.log(renderReadmeTable(record));
+} else {
+  const table = renderReadmeTable(record);
+  const updated = `${readme.slice(0, start + README_START.length)}\n\n${table}\n\n${readme.slice(end)}`;
+  await writeFile(README_PATH, updated);
+  console.log(`updated benchmark table in ${README_PATH}`);
+}
diff --git a/packages/evals/scripts/run.ts b/packages/evals/scripts/run.ts
new file mode 100644
index 0000000..c039feb
--- /dev/null
+++ b/packages/evals/scripts/run.ts
@@ -0,0 +1,58 @@
+// Runs the benchmark: every pinned document through every model, raw results
+// serialized to results/ so reports are reproducible from the run record.
+import { access, mkdir, writeFile } from 'node:fs/promises';
+import { join } from 'node:path';
+import { loadCordDocuments } from '../src/datasets/cord-source.js';
+import { loadDocileDocuments } from '../src/datasets/docile-source.js';
+import { loadManifest } from '../src/manifest.js';
+import { benchmarkModels } from '../src/models.js';
+import { runModel } from '../src/runner.js';
+import type { EvalDocument, RunRecord } from '../src/types.js';
+import { CORD_PARQUET_PATH, DOCILE_ROOT, RESULTS_DIR } from './paths.js';
+
+async function exists(path: string): Promise<boolean> {
+  try {
+    await access(path);
+    return true;
+  } catch {
+    return false;
+  }
+}
+
+const { manifest, checksum } = await loadManifest();
+const models = benchmarkModels();
+console.log(`models under test: ${models.map((m) => m.name).join(', ')}`);
+
+const docs: EvalDocument[] = [];
+if (manifest.cord.docs.length > 0) {
+  if (!(await exists(CORD_PARQUET_PATH))) throw new Error('cord parquet missing — run `pnpm fetch-data` first');
+  docs.push(...(await loadCordDocuments(manifest, CORD_PARQUET_PATH)));
+}
+if (manifest.docile.docs.length > 0) {
+  if (!(await exists(join(DOCILE_ROOT, 'annotations')))) {
+    throw new Error('docile docs are pinned but the dataset is not fetched — run `pnpm fetch-data` with DOCILE_TOKEN');
+  }
+  docs.push(...(await loadDocileDocuments(manifest, DOCILE_ROOT)));
+} else {
+  console.log('docile: no pinned docs yet; running the receipt half only');
+}
+if (docs.length === 0) throw new Error('no benchmark documents to run — run `pnpm fetch-data` and `pnpm pin` first');
+
+const startedAt = new Date().toISOString();
+const record: RunRecord = { startedAt, manifestChecksum: checksum, runs: [] };
+
+for (const model of models) {
+  console.log(`\n=== ${model.name} — ${docs.length} docs ===`);
+  const run = await runModel(model, docs, {
+    onDocDone: (r) => {
+      const correct = r.fields.filter((f) => f.valueCorrect).length;
+      console.log(`  ${r.docId}: ${correct}/${r.fields.length} fields${r.error !== null ? ` (${r.error})` : ''}`);
+    },
+  });
+  record.runs.push(run);
+}
+
+await mkdir(RESULTS_DIR, { recursive: true });
+const outPath = join(RESULTS_DIR, `run-${startedAt.replace(/[:.]/g, '-')}.json`);
+await writeFile(outPath, `${JSON.stringify(record, null, 2)}\n`);
+console.log(`\nrun record written to ${outPath}\nGenerate the report with: pnpm report ${outPath}`);
diff --git a/packages/evals/src/datasets/cord-source.ts b/packages/evals/src/datasets/cord-source.ts
new file mode 100644
index 0000000..e7a3acd
--- /dev/null
+++ b/packages/evals/src/datasets/cord-source.ts
@@ -0,0 +1,67 @@
+import { asyncBufferFromFile, parquetMetadataAsync, parquetReadObjects } from 'hyparquet';
+import { sniffMediaType } from 'extractkit';
+import { sha256Hex, type CordPin, type Manifest } from '../manifest.js';
+import { cordToGroundTruth, type CordGroundTruth } from './cord.js';
+import type { EvalDocument } from '../types.js';
+
+export interface CordRow {
+  rowIndex: number;
+  imageBytes: Uint8Array;
+  groundTruth: CordGroundTruth;
+}
+
+/** Read specific rows (or all rows) of a CORD-v2 split parquet. */
+export async function readCordRows(parquetPath: string, rowIndices?: number[]): Promise<CordRow[]> {
+  const file = await asyncBufferFromFile(parquetPath);
+  const metadata = await parquetMetadataAsync(file);
+  const total = Number(metadata.num_rows);
+  const indices = rowIndices ?? Array.from({ length: total }, (_, i) => i);
+  const rows: CordRow[] = [];
+  for (const rowIndex of indices) {
+    if (rowIndex < 0 || rowIndex >= total) throw new Error(`Row ${rowIndex} out of range (parquet has ${total} rows)`);
+    // utf8:false keeps image bytes binary; UTF8-annotated columns still decode.
+    const [row] = await parquetReadObjects({ file, metadata, rowStart: rowIndex, rowEnd: rowIndex + 1, utf8: false });
+    const record = row as { image: { bytes: Uint8Array }; ground_truth: string };
+    rows.push({
+      rowIndex,
+      imageBytes: record.image.bytes,
+      groundTruth: JSON.parse(record.ground_truth) as CordGroundTruth,
+    });
+  }
+  return rows;
+}
+
+/** Load the pinned CORD documents from a locally fetched parquet, verifying
+ * every pinned checksum. */
+export async function loadCordDocuments(manifest: Manifest, parquetPath: string): Promise<EvalDocument[]> {
+  const pins = manifest.cord.docs;
+  const byIndex = new Map<number, CordPin>(pins.map((p) => [p.rowIndex, p]));
+  const rows = await readCordRows(parquetPath, pins.map((p) => p.rowIndex));
+  return rows.map((row) => {
+    const pin = byIndex.get(row.rowIndex) as CordPin;
+    const digest = sha256Hex(row.imageBytes);
+    if (digest !== pin.imageSha256) {
+      throw new Error(
+        `cord row ${row.rowIndex}: image sha256 ${digest} does not match pinned ${pin.imageSha256} — dataset content changed?`,
+      );
+    }
+    if (row.groundTruth.meta.image_id !== pin.imageId) {
+      throw new Error(
+        `cord row ${row.rowIndex}: image_id ${row.groundTruth.meta.image_id} does not match pinned ${pin.imageId}`,
+      );
+    }
+    const mediaType = sniffMediaType(row.imageBytes);
+    if (mediaType === null) throw new Error(`cord row ${row.rowIndex}: unrecognized image format`);
+    const { fields, lineItemCount } = cordToGroundTruth(row.groundTruth);
+    return {
+      id: `cord/${manifest.cord.split}/${row.rowIndex}`,
+      dataset: 'cord' as const,
+      schema: 'receipt' as const,
+      bytes: row.imageBytes,
+      mediaType,
+      pages: 1,
+      fields,
+      lineItemCount,
+    };
+  });
+}
diff --git a/packages/evals/src/datasets/cord.ts b/packages/evals/src/datasets/cord.ts
new file mode 100644
index 0000000..0df24d4
--- /dev/null
+++ b/packages/evals/src/datasets/cord.ts
@@ -0,0 +1,264 @@
+import type { BBox } from 'extractkit';
+import { valuesMatch } from '../normalize.js';
+import type { CompareKind, GroundTruthField, GroundTruthRegion } from '../types.js';
+
+/** CORD-v2 per-document ground truth, as stored in the `ground_truth` column. */
+export interface CordGroundTruth {
+  gt_parse: CordParse;
+  meta: { image_id: number; image_size: { width: number; height: number } };
+  valid_line: CordLine[];
+}
+
+interface CordParse {
+  menu?: CordMenuItem | CordMenuItem[];
+  sub_total?: Record<string, unknown>;
+  total?: Record<string, unknown>;
+  [key: string]: unknown;
+}
+
+interface CordMenuItem {
+  nm?: string;
+  cnt?: string;
+  unitprice?: string;
+  price?: string;
+  sub?: CordMenuItem | CordMenuItem[];
+  [key: string]: unknown;
+}
+
+interface CordLine {
+  category: string;
+  group_id: number;
+  sub_group_id?: number;
+  words: Array<{
+    text: string;
+    quad: { x1: number; y1: number; x2: number; y2: number; x3: number; y3: number; x4: number; y4: number };
+  }>;
+}
+
+/** Thrown when a document's gt_parse cannot be mapped confidently onto the
+ * receipt schema; the pin script skips such documents. */
+export class CordMappingError extends Error {}
+
+function asArray<T>(value: T | T[] | undefined): T[] {
+  if (value === undefined) return [];
+  return Array.isArray(value) ? value : [value];
+}
+
+function itemValue(item: CordMenuItem, key: 'nm' | 'cnt' | 'unitprice' | 'price'): string | null {
+  const value = item[key];
+  if (value === undefined || value === null) return null;
+  if (typeof value !== 'string') {
+    throw new CordMappingError(`menu ${key}: expected a string, got ${Array.isArray(value) ? 'array' : typeof value}`);
+  }
+  return value;
+}
+
+function scalarValue(parse: CordParse, section: string, key: string): string | null {
+  const record = parse[section];
+  if (record === undefined || record === null) return null;
+  if (typeof record !== 'object' || Array.isArray(record)) {
+    throw new CordMappingError(`${section}: expected an object`);
+  }
+  const value = (record as Record<string, unknown>)[key];
+  if (value === undefined || value === null) return null;
+  if (typeof value !== 'string') {
+    throw new CordMappingError(`${section}.${key}: expected a string, got ${Array.isArray(value) ? 'array' : typeof value}`);
+  }
+  return value;
+}
+
+type CordWord = CordLine['words'][number];
+
+function unionBBox(words: CordWord[], width: number, height: number): BBox {
+  let x0 = Infinity;
+  let y0 = Infinity;
+  let x1 = -Infinity;
+  let y1 = -Infinity;
+  for (const { quad } of words) {
+    x0 = Math.min(x0, quad.x1, quad.x2, quad.x3, quad.x4);
+    x1 = Math.max(x1, quad.x1, quad.x2, quad.x3, quad.x4);
+    y0 = Math.min(y0, quad.y1, quad.y2, quad.y3, quad.y4);
+    y1 = Math.max(y1, quad.y1, quad.y2, quad.y3, quad.y4);
+  }
+  const clamp = (v: number, max: number) => Math.min(Math.max(v / max, 0), 1);
+  return { x0: clamp(x0, width), y0: clamp(y0, height), x1: clamp(x1, width), y1: clamp(y1, height) };
+}
+
+function joinWords(words: CordWord[]): string {
+  return words
+    .map((w) => w.text)
+    .join(' ')
+    .trim();
+}
+
+/**
+ * The word spans that carry `value` within the annotated line(s). CORD lines
+ * often include label words around the value ("PB1 10% 3,650" for tax_price
+ * "3,650"), so we accept every word suffix whose text matches the value —
+ * each is an acceptable grounding region. Empty when no suffix matches,
+ * which means the annotation cannot be reconciled with gt_parse.
+ */
+function matchingSuffixes(words: CordWord[], value: string, compare: CompareKind): CordWord[][] {
+  const spans: CordWord[][] = [];
+  for (let k = 1; k <= words.length; k++) {
+    const span = words.slice(words.length - k);
+    if (valuesMatch(value, joinWords(span), compare)) spans.push(span);
+  }
+  return spans;
+}
+
+/** A flattened line item paired with the valid_line selector that owns its
+ * annotation lines. Sub-items (`menu.sub.*` categories) become their own
+ * line items, in printed order after their parent. */
+interface FlatItem {
+  values: { nm: string | null; cnt: string | null; unitprice: string | null; price: string | null };
+  categoryPrefix: 'menu' | 'menu.sub';
+  groupId: number;
+  subGroupId: number | null;
+}
+
+const ITEM_FIELDS = [
+  { key: 'nm', path: 'description', compare: 'text' },
+  { key: 'cnt', path: 'quantity', compare: 'count' },
+  { key: 'unitprice', path: 'unitPrice', compare: 'money' },
+  { key: 'price', path: 'amount', compare: 'money' },
+] as const;
+
+const SCALAR_FIELDS = [
+  { section: 'sub_total', key: 'subtotal_price', path: 'subtotal', compare: 'money' },
+  { section: 'sub_total', key: 'discount_price', path: 'discount', compare: 'money' },
+  { section: 'sub_total', key: 'service_price', path: 'serviceCharge', compare: 'money' },
+  { section: 'sub_total', key: 'tax_price', path: 'tax', compare: 'money' },
+  { section: 'total', key: 'total_price', path: 'total', compare: 'money' },
+] as const;
+
+/**
+ * Flatten gt_parse menu items and pair each with its valid_line group.
+ * Pairing assumes ascending group_id (and sub_group_id within a group)
+ * follows printed order, which matches gt_parse order; the text-consistency
+ * check in fieldRegions() rejects any document where that assumption fails.
+ */
+function flattenItems(gt: CordGroundTruth): FlatItem[] {
+  const items = asArray(gt.gt_parse.menu);
+  const parentGroupIds = [
+    ...new Set(gt.valid_line.filter((l) => l.category === 'menu.nm').map((l) => l.group_id)),
+  ].sort((a, b) => a - b);
+  if (parentGroupIds.length !== items.length) {
+    throw new CordMappingError(
+      `menu has ${items.length} gt_parse items but ${parentGroupIds.length} annotated menu.nm groups`,
+    );
+  }
+  const flat: FlatItem[] = [];
+  items.forEach((item, i) => {
+    const groupId = parentGroupIds[i] as number;
+    flat.push({
+      values: {
+        nm: itemValue(item, 'nm'),
+        cnt: itemValue(item, 'cnt'),
+        unitprice: itemValue(item, 'unitprice'),
+        price: itemValue(item, 'price'),
+      },
+      categoryPrefix: 'menu',
+      groupId,
+      subGroupId: null,
+    });
+    const subs = asArray(item.sub);
+    if (subs.length === 0) return;
+    const subGroupIds = [
+      ...new Set(
+        gt.valid_line
+          .filter((l) => l.category.startsWith('menu.sub.') && l.group_id === groupId)
+          .map((l) => l.sub_group_id ?? 0),
+      ),
+    ].sort((a, b) => a - b);
+    if (subGroupIds.length !== subs.length) {
+      throw new CordMappingError(
+        `menu item ${i} has ${subs.length} sub-items but ${subGroupIds.length} annotated sub-groups`,
+      );
+    }
+    subs.forEach((sub, j) => {
+      flat.push({
+        values: {
+          nm: itemValue(sub, 'nm'),
+          cnt: itemValue(sub, 'cnt'),
+          unitprice: itemValue(sub, 'unitprice'),
+          price: itemValue(sub, 'price'),
+        },
+        categoryPrefix: 'menu.sub',
+        groupId,
+        subGroupId: subGroupIds[j] as number,
+      });
+    });
+  });
+  return flat;
+}
+
+/**
+ * Grounding regions for one gt_parse value: the matching word suffix(es) of
+ * the valid_line entries selected by category/group. Empty when no line
+ * carries the category (some gt_parse values have no annotated region);
+ * throws when lines exist but none of their word suffixes carries the value —
+ * that means the pairing misfired and the document should not be pinned.
+ */
+function fieldRegions(
+  gt: CordGroundTruth,
+  category: string,
+  value: string,
+  compare: CompareKind,
+  groupId: number | null,
+  subGroupId: number | null,
+): GroundTruthRegion[] {
+  const lines = gt.valid_line.filter(
+    (l) =>
+      l.category === category &&
+      (groupId === null || l.group_id === groupId) &&
+      (subGroupId === null || (l.sub_group_id ?? 0) === subGroupId),
+  );
+  if (lines.length === 0) return [];
+  const words = lines.flatMap((l) => l.words);
+  const spans = matchingSuffixes(words, value, compare);
+  if (spans.length === 0) {
+    throw new CordMappingError(
+      `${category}: annotated text ${JSON.stringify(joinWords(words))} does not carry gt_parse value ${JSON.stringify(value)}`,
+    );
+  }
+  const { width, height } = gt.meta.image_size;
+  return spans.map((span) => ({ page: 0, bbox: unionBBox(span, width, height) }));
+}
+
+/**
+ * Map one CORD-v2 ground-truth record onto receipt-schema ground truth.
+ * Throws CordMappingError when the record cannot be mapped confidently;
+ * such documents are excluded at pin time, never silently mis-scored.
+ */
+export function cordToGroundTruth(gt: CordGroundTruth): { fields: GroundTruthField[]; lineItemCount: number } {
+  const fields: GroundTruthField[] = [];
+  const items = flattenItems(gt);
+
+  items.forEach((item, i) => {
+    for (const { key, path, compare } of ITEM_FIELDS) {
+      const value = item.values[key];
+      fields.push({
+        path: `lineItems.${i}.${path}`,
+        value,
+        compare,
+        regions:
+          value === null
+            ? []
+            : fieldRegions(gt, `${item.categoryPrefix}.${key}`, value, compare, item.groupId, item.subGroupId),
+      });
+    }
+  });
+
+  for (const { section, key, path, compare } of SCALAR_FIELDS) {
+    const value = scalarValue(gt.gt_parse, section, key);
+    fields.push({
+      path,
+      value,
+      compare,
+      regions: value === null ? [] : fieldRegions(gt, `${section}.${key}`, value, compare, null, null),
+    });
+  }
+
+  return { fields, lineItemCount: items.length };
+}
diff --git a/packages/evals/src/datasets/docile-source.ts b/packages/evals/src/datasets/docile-source.ts
new file mode 100644
index 0000000..9a7d06c
--- /dev/null
+++ b/packages/evals/src/datasets/docile-source.ts
@@ -0,0 +1,52 @@
+import { readFile } from 'node:fs/promises';
+import { join } from 'node:path';
+import { sha256Hex, type Manifest } from '../manifest.js';
+import { docileToGroundTruth, type DocileAnnotation } from './docile.js';
+import type { EvalDocument } from '../types.js';
+
+/** Read one document from an extracted DocILE dataset directory
+ * (`<root>/pdfs/{docId}.pdf` + `<root>/annotations/{docId}.json`). */
+export async function readDocileDoc(
+  root: string,
+  docId: string,
+): Promise<{ pdfBytes: Uint8Array; annotation: DocileAnnotation }> {
+  const pdfBytes = new Uint8Array(await readFile(join(root, 'pdfs', `${docId}.pdf`)));
+  const annotation = JSON.parse(await readFile(join(root, 'annotations', `${docId}.json`), 'utf8')) as DocileAnnotation;
+  return { pdfBytes, annotation };
+}
+
+/** Doc ids in a DocILE split (`<root>/{split}.json` is a JSON array). */
+export async function readDocileSplit(root: string, split: string): Promise<string[]> {
+  return JSON.parse(await readFile(join(root, `${split}.json`), 'utf8')) as string[];
+}
+
+/** Load the pinned DocILE documents from a locally downloaded dataset,
+ * verifying every pinned checksum. */
+export async function loadDocileDocuments(manifest: Manifest, root: string): Promise<EvalDocument[]> {
+  return Promise.all(
+    manifest.docile.docs.map(async (pin) => {
+      const { pdfBytes, annotation } = await readDocileDoc(root, pin.docId);
+      const pdfDigest = sha256Hex(pdfBytes);
+      if (pdfDigest !== pin.pdfSha256) {
+        throw new Error(`docile ${pin.docId}: pdf sha256 ${pdfDigest} does not match pinned ${pin.pdfSha256}`);
+      }
+      const annotationDigest = sha256Hex(await readFile(join(root, 'annotations', `${pin.docId}.json`)));
+      if (annotationDigest !== pin.annotationSha256) {
+        throw new Error(
+          `docile ${pin.docId}: annotation sha256 ${annotationDigest} does not match pinned ${pin.annotationSha256}`,
+        );
+      }
+      const { fields, lineItemCount } = docileToGroundTruth(annotation);
+      return {
+        id: `docile/${manifest.docile.split}/${pin.docId}`,
+        dataset: 'docile' as const,
+        schema: 'invoice' as const,
+        bytes: pdfBytes,
+        mediaType: 'application/pdf' as const,
+        pages: annotation.metadata.page_count,
+        fields,
+        lineItemCount,
+      };
+    }),
+  );
+}
diff --git a/packages/evals/src/datasets/docile.ts b/packages/evals/src/datasets/docile.ts
new file mode 100644
index 0000000..e9e311e
--- /dev/null
+++ b/packages/evals/src/datasets/docile.ts
@@ -0,0 +1,151 @@
+import type { BBox } from 'extractkit';
+import { normalizeValue } from '../normalize.js';
+import type { CompareKind, GroundTruthField, GroundTruthRegion } from '../types.js';
+
+/**
+ * DocILE annotation file (`<root>/annotations/{docid}.json`) as shipped in
+ * the annotated-trainval archive. bbox is [left, top, right, bottom] in
+ * page-relative 0–1 coordinates, origin top-left; page is 0-based — the same
+ * conventions extractkit uses.
+ */
+export interface DocileAnnotation {
+  field_extractions: DocileField[];
+  line_item_extractions: DocileField[];
+  metadata: {
+    page_count: number;
+    document_type: string;
+    currency: string;
+    [key: string]: unknown;
+  };
+}
+
+export interface DocileField {
+  fieldtype: string;
+  text: string;
+  page: number;
+  bbox: [number, number, number, number];
+  line_item_id?: number;
+}
+
+/** Thrown when an annotation cannot be mapped confidently onto the invoice
+ * schema; the pin script skips such documents. */
+export class DocileMappingError extends Error {}
+
+const HEADER_FIELDS = [
+  { fieldtype: 'vendor_name', path: 'vendorName', compare: 'text' },
+  { fieldtype: 'document_id', path: 'invoiceNumber', compare: 'text' },
+  { fieldtype: 'date_issue', path: 'issueDate', compare: 'text' },
+  { fieldtype: 'date_due', path: 'dueDate', compare: 'text' },
+  { fieldtype: 'currency_code_amount_due', path: 'currency', compare: 'text' },
+  { fieldtype: 'amount_total_net', path: 'subtotal', compare: 'money' },
+  { fieldtype: 'amount_total_tax', path: 'tax', compare: 'money' },
+  { fieldtype: 'amount_total_gross', path: 'total', compare: 'money' },
+] as const;
+
+/** Net preferred over gross so "subtotal + tax = total" stays coherent with
+ * the header mapping (subtotal ← amount_total_net). */
+const ITEM_FIELDS = [
+  { fieldtypes: ['line_item_description'], path: 'description', compare: 'text' },
+  { fieldtypes: ['line_item_quantity'], path: 'quantity', compare: 'count' },
+  { fieldtypes: ['line_item_unit_price_net', 'line_item_unit_price_gross'], path: 'unitPrice', compare: 'money' },
+  { fieldtypes: ['line_item_amount_net', 'line_item_amount_gross'], path: 'amount', compare: 'money' },
+] as const;
+
+function toRegion(field: DocileField): GroundTruthRegion {
+  const [x0, y0, x1, y1] = field.bbox;
+  const bbox: BBox = { x0, y0, x1, y1 };
+  return { page: field.page, bbox };
+}
+
+function byReadingOrder(a: DocileField, b: DocileField): number {
+  return a.page - b.page || a.bbox[1] - b.bbox[1] || a.bbox[0] - b.bbox[0];
+}
+
+/**
+ * Collapse every annotated occurrence of one header fieldtype into a single
+ * ground-truth field: the first occurrence (reading order) is canonical,
+ * occurrences printed differently become altValues, and all boxes are
+ * acceptable grounding regions.
+ */
+function headerField(
+  occurrences: DocileField[],
+  path: string,
+  compare: CompareKind,
+): GroundTruthField {
+  if (occurrences.length === 0) return { path, value: null, compare, regions: [] };
+  const sorted = [...occurrences].sort(byReadingOrder);
+  const first = sorted[0] as DocileField;
+  const canonical = first.text;
+  const seen = new Set([normalizeValue(canonical, compare)]);
+  const altValues: string[] = [];
+  for (const occ of sorted.slice(1)) {
+    const normalized = normalizeValue(occ.text, compare);
+    if (!seen.has(normalized)) {
+      seen.add(normalized);
+      altValues.push(occ.text);
+    }
+  }
+  return {
+    path,
+    value: canonical,
+    ...(altValues.length > 0 ? { altValues } : {}),
+    compare,
+    regions: sorted.map(toRegion),
+  };
+}
+
+/** One line-item field: fragments of the same fieldtype (multi-line values)
+ * are joined in reading order; distinct-text duplicates are ambiguous and
+ * reject the document. */
+function itemField(
+  occurrences: DocileField[],
+  path: string,
+  compare: CompareKind,
+): GroundTruthField {
+  if (occurrences.length === 0) return { path, value: null, compare, regions: [] };
+  const sorted = [...occurrences].sort(byReadingOrder);
+  const value = sorted.map((f) => f.text).join(' ');
+  return { path, value, compare, regions: sorted.map(toRegion) };
+}
+
+/**
+ * Map one DocILE annotation onto invoice-schema ground truth. Line items
+ * are ordered by the reading order of their topmost annotation, matching
+ * the "in printed order" instruction in the invoice schema. Throws
+ * DocileMappingError when the annotation cannot be mapped confidently.
+ */
+export function docileToGroundTruth(annotation: DocileAnnotation): {
+  fields: GroundTruthField[];
+  lineItemCount: number;
+} {
+  const fields: GroundTruthField[] = [];
+
+  for (const { fieldtype, path, compare } of HEADER_FIELDS) {
+    const occurrences = annotation.field_extractions.filter((f) => f.fieldtype === fieldtype);
+    fields.push(headerField(occurrences, path, compare));
+  }
+
+  const byItem = new Map<number, DocileField[]>();
+  for (const field of annotation.line_item_extractions) {
+    if (field.line_item_id === undefined) {
+      throw new DocileMappingError(`line_item_extractions entry ${field.fieldtype} has no line_item_id`);
+    }
+    const list = byItem.get(field.line_item_id) ?? [];
+    list.push(field);
+    byItem.set(field.line_item_id, list);
+  }
+
+  const itemOrder = [...byItem.entries()]
+    .map(([id, list]) => ({ id, list, first: [...list].sort(byReadingOrder)[0] as DocileField }))
+    .sort((a, b) => byReadingOrder(a.first, b.first));
+
+  itemOrder.forEach(({ list }, i) => {
+    for (const { fieldtypes, path, compare } of ITEM_FIELDS) {
+      // Prefer net over gross: use the first fieldtype that has occurrences.
+      const present = fieldtypes.map((ft) => list.filter((f) => f.fieldtype === ft)).find((occ) => occ.length > 0) ?? [];
+      fields.push({ ...itemField(present, path, compare), path: `lineItems.${i}.${path}` });
+    }
+  });
+
+  return { fields, lineItemCount: itemOrder.length };
+}
diff --git a/packages/evals/src/index.ts b/packages/evals/src/index.ts
new file mode 100644
index 0000000..78b8b82
--- /dev/null
+++ b/packages/evals/src/index.ts
@@ -0,0 +1,29 @@
+export { schemas, invoiceSchema, receiptSchema } from './schemas.js';
+export type { Invoice, Receipt } from './schemas.js';
+export { normalizeCount, normalizeMoney, normalizeText, normalizeValue, valuesMatch } from './normalize.js';
+export { iou, leafAt, scoreExtraction, scoreFailure, summarizeRun } from './metrics.js';
+export type { ModelSummary, SchemaSummary } from './metrics.js';
+export { runModel } from './runner.js';
+export type { RunOptions } from './runner.js';
+export { loadManifest, sha256Hex, MANIFEST_PATH } from './manifest.js';
+export type { CordPin, DocilePin, Manifest } from './manifest.js';
+export { cordToGroundTruth, CordMappingError } from './datasets/cord.js';
+export type { CordGroundTruth } from './datasets/cord.js';
+export { loadCordDocuments, readCordRows } from './datasets/cord-source.js';
+export { docileToGroundTruth, DocileMappingError } from './datasets/docile.js';
+export type { DocileAnnotation, DocileField } from './datasets/docile.js';
+export { loadDocileDocuments, readDocileDoc, readDocileSplit } from './datasets/docile-source.js';
+export { renderBenchmarkPage, renderReadmeTable } from './report.js';
+export type {
+  CompareKind,
+  DatasetId,
+  DocResult,
+  EvalDocument,
+  EvalModel,
+  FieldResult,
+  GroundTruthField,
+  GroundTruthRegion,
+  ModelRun,
+  RunRecord,
+  SchemaId,
+} from './types.js';
diff --git a/packages/evals/src/manifest.ts b/packages/evals/src/manifest.ts
new file mode 100644
index 0000000..b5e05e8
--- /dev/null
+++ b/packages/evals/src/manifest.ts
@@ -0,0 +1,59 @@
+import { createHash } from 'node:crypto';
+import { readFile } from 'node:fs/promises';
+import { fileURLToPath } from 'node:url';
+
+/**
+ * The pinned benchmark selection. Documents are never vendored: the manifest
+ * holds ids + checksums, and the fetch script pulls the documents from their
+ * canonical hosts. Every published number is computed on exactly this set.
+ */
+export interface Manifest {
+  version: 1;
+  cord: {
+    dataset: string;
+    /** Git revision of the dataset repo the parquet is fetched at. */
+    revision: string;
+    /** Repo-relative parquet path holding the split. */
+    file: string;
+    /** SHA-256 of the parquet file. */
+    sha256: string;
+    split: string;
+    docs: CordPin[];
+  };
+  docile: {
+    /** Archive name passed to the DocILE downloader. */
+    archive: string;
+    split: string;
+    docs: DocilePin[];
+  };
+}
+
+export interface CordPin {
+  /** Row index within the split parquet. */
+  rowIndex: number;
+  /** image_id from the row's ground-truth metadata. */
+  imageId: number;
+  /** SHA-256 of the row's image bytes. */
+  imageSha256: string;
+}
+
+export interface DocilePin {
+  docId: string;
+  /** SHA-256 of `pdfs/{docId}.pdf`. */
+  pdfSha256: string;
+  /** SHA-256 of `annotations/{docId}.json`. */
+  annotationSha256: string;
+}
+
+export const MANIFEST_PATH = fileURLToPath(new URL('../data/manifest.json', import.meta.url));
+
+export function sha256Hex(bytes: Uint8Array | string): string {
+  return createHash('sha256').update(bytes).digest('hex');
+}
+
+export async function loadManifest(path: string = MANIFEST_PATH): Promise<{ manifest: Manifest; checksum: string }> {
+  const raw = await readFile(path, 'utf8');
+  const manifest = JSON.parse(raw) as Manifest;
+  if (manifest.version !== 1) throw new Error(`Unsupported manifest version: ${String(manifest.version)}`);
+  return { manifest, checksum: sha256Hex(raw) };
+}
diff --git a/packages/evals/src/metrics.ts b/packages/evals/src/metrics.ts
new file mode 100644
index 0000000..753d1d1
--- /dev/null
+++ b/packages/evals/src/metrics.ts
@@ -0,0 +1,183 @@
+import type { BBox, ExtractedField } from 'extractkit';
+import { valuesMatch } from './normalize.js';
+import type { DocResult, EvalDocument, FieldResult, GroundTruthField, ModelRun, SchemaId } from './types.js';
+
+export function iou(a: BBox, b: BBox): number {
+  const ix = Math.max(0, Math.min(a.x1, b.x1) - Math.max(a.x0, b.x0));
+  const iy = Math.max(0, Math.min(a.y1, b.y1) - Math.max(a.y0, b.y0));
+  const inter = ix * iy;
+  const areaA = Math.max(0, a.x1 - a.x0) * Math.max(0, a.y1 - a.y0);
+  const areaB = Math.max(0, b.x1 - b.x0) * Math.max(0, b.y1 - b.y0);
+  const union = areaA + areaB - inter;
+  return union <= 0 ? 0 : inter / union;
+}
+
+function isExtractedField(node: unknown): node is ExtractedField<unknown> {
+  return (
+    typeof node === 'object' &&
+    node !== null &&
+    'value' in node &&
+    'confidence' in node &&
+    'page' in node &&
+    'bbox' in node
+  );
+}
+
+/** Walk an extract() FieldMap by dot path; null when the path leads nowhere
+ * (e.g. the model returned fewer line items than ground truth has). */
+export function leafAt(fields: unknown, path: string): ExtractedField<unknown> | null {
+  let node: unknown = fields;
+  for (const segment of path.split('.')) {
+    if (node === null || typeof node !== 'object') return null;
+    node = (node as Record<string, unknown>)[segment];
+  }
+  return isExtractedField(node) ? node : null;
+}
+
+function scoreField(gt: GroundTruthField, leaf: ExtractedField<unknown> | null): FieldResult {
+  const predicted = leaf === null || leaf.value === null ? null : String(leaf.value);
+  const valueCorrect =
+    valuesMatch(gt.value, predicted, gt.compare) ||
+    (predicted !== null && (gt.altValues ?? []).some((alt) => valuesMatch(alt, predicted, gt.compare)));
+
+  let iouScore: number | null = null;
+  if (valueCorrect && gt.regions.length > 0) {
+    const samePage = gt.regions.filter((r) => leaf !== null && r.page === leaf.page);
+    iouScore =
+      leaf === null || leaf.bbox === null || samePage.length === 0
+        ? 0
+        : Math.max(...samePage.map((r) => iou(r.bbox, leaf.bbox as BBox)));
+  }
+
+  return {
+    path: gt.path,
+    compare: gt.compare,
+    expected: gt.value,
+    predicted,
+    valueCorrect,
+    iou: iouScore,
+    predictedPage: leaf?.page ?? null,
+  };
+}
+
+/** Number of line items the model returned, read from the field map. */
+function predictedLineItemCount(fields: unknown): number {
+  const items = (fields as Record<string, unknown> | null)?.['lineItems'];
+  return Array.isArray(items) ? items.length : 0;
+}
+
+/** Score one successful extraction against a document's ground truth. */
+export function scoreExtraction(
+  doc: EvalDocument,
+  extraction: { fields: unknown },
+): Pick<DocResult, 'fields' | 'extraLineItems'> {
+  return {
+    fields: doc.fields.map((gt) => scoreField(gt, leafAt(extraction.fields, gt.path))),
+    extraLineItems: Math.max(0, predictedLineItemCount(extraction.fields) - doc.lineItemCount),
+  };
+}
+
+/** A failed extraction scores every ground-truth field as incorrect. */
+export function scoreFailure(doc: EvalDocument): Pick<DocResult, 'fields' | 'extraLineItems'> {
+  return {
+    fields: doc.fields.map((gt) => ({
+      path: gt.path,
+      compare: gt.compare,
+      expected: gt.value,
+      predicted: null,
+      valueCorrect: false,
+      iou: gt.regions.length > 0 ? 0 : null,
+      predictedPage: null,
+    })),
+    extraLineItems: 0,
+  };
+}
+
+export interface SchemaSummary {
+  docs: number;
+  failedDocs: number;
+  /** Fraction of ground-truth fields whose predicted value matched. */
+  fieldAccuracy: number;
+  fieldsScored: number;
+  grounding: {
+    /** Fields with a correct value and a ground-truth region. */
+    scoreable: number;
+    meanIoU: number | null;
+    /** Fraction of scoreable fields with IoU ≥ 0.5. */
+    hitAt50: number | null;
+  };
+  /** Accuracy per schema field, line items collapsed across indices. */
+  perField: Array<{ field: string; accuracy: number; count: number }>;
+  extraLineItems: number;
+  cost: {
+    totalUSD: number | null;
+    per1kDocsUSD: number | null;
+    inputTokens: number;
+    outputTokens: number;
+  };
+}
+
+export interface ModelSummary {
+  model: string;
+  perSchema: Partial<Record<SchemaId, SchemaSummary>>;
+}
+
+/** Collapse line-item indices: `lineItems.3.amount` → `lineItems[].amount`. */
+function fieldKey(path: string): string {
+  return path.replace(/\.\d+\./g, '[].');
+}
+
+export function summarizeRun(run: ModelRun): ModelSummary {
+  const perSchema: Partial<Record<SchemaId, SchemaSummary>> = {};
+  const bySchema = new Map<SchemaId, DocResult[]>();
+  for (const doc of run.docs) {
+    const list = bySchema.get(doc.schema) ?? [];
+    list.push(doc);
+    bySchema.set(doc.schema, list);
+  }
+
+  for (const [schema, docs] of bySchema) {
+    const fields = docs.flatMap((d) => d.fields);
+    const correct = fields.filter((f) => f.valueCorrect).length;
+    const groundable = fields.filter((f) => f.iou !== null);
+    const perFieldMap = new Map<string, { correct: number; count: number }>();
+    for (const f of fields) {
+      const key = fieldKey(f.path);
+      const entry = perFieldMap.get(key) ?? { correct: 0, count: 0 };
+      entry.count += 1;
+      if (f.valueCorrect) entry.correct += 1;
+      perFieldMap.set(key, entry);
+    }
+
+    const costs = docs.map((d) => d.usage?.costUSD ?? null);
+    const totalUSD = costs.every((c) => c === null) ? null : costs.reduce<number>((sum, c) => sum + (c ?? 0), 0);
+
+    perSchema[schema] = {
+      docs: docs.length,
+      failedDocs: docs.filter((d) => d.error !== null).length,
+      fieldAccuracy: fields.length === 0 ? 0 : correct / fields.length,
+      fieldsScored: fields.length,
+      grounding: {
+        scoreable: groundable.length,
+        meanIoU:
+          groundable.length === 0
+            ? null
+            : groundable.reduce((sum, f) => sum + (f.iou as number), 0) / groundable.length,
+        hitAt50:
+          groundable.length === 0 ? null : groundable.filter((f) => (f.iou as number) >= 0.5).length / groundable.length,
+      },
+      perField: [...perFieldMap.entries()]
+        .map(([field, { correct: c, count }]) => ({ field, accuracy: c / count, count }))
+        .sort((a, b) => a.field.localeCompare(b.field)),
+      extraLineItems: docs.reduce((sum, d) => sum + d.extraLineItems, 0),
+      cost: {
+        totalUSD,
+        per1kDocsUSD: totalUSD === null || docs.length === 0 ? null : (totalUSD / docs.length) * 1000,
+        inputTokens: docs.reduce((sum, d) => sum + (d.usage?.inputTokens ?? 0), 0),
+        outputTokens: docs.reduce((sum, d) => sum + (d.usage?.outputTokens ?? 0), 0),
+      },
+    };
+  }
+
+  return { model: run.model, perSchema };
+}
diff --git a/packages/evals/src/models.ts b/packages/evals/src/models.ts
new file mode 100644
index 0000000..eda9be8
--- /dev/null
+++ b/packages/evals/src/models.ts
@@ -0,0 +1,111 @@
+import { anthropic } from '@ai-sdk/anthropic';
+import { google } from '@ai-sdk/google';
+import { openai } from '@ai-sdk/openai';
+import type { LanguageModel } from 'ai';
+import type { Pricing } from 'extractkit';
+import type { EvalModel } from './types.js';
+
+export type Provider = 'anthropic' | 'openai' | 'google';
+
+export const PROVIDERS: readonly Provider[] = ['anthropic', 'openai', 'google'];
+
+interface CatalogEntry {
+  /** Display name used in reports; also the provider's model id. */
+  name: string;
+  pricing: Pricing;
+}
+
+interface ProviderSpec {
+  /** Env var the AI SDK provider reads for its API key. */
+  apiKeyEnv: string;
+  create: (modelId: string) => LanguageModel;
+  models: CatalogEntry[];
+}
+
+/**
+ * The benchmark lineup: three tiers per provider (flagship → cost-efficient),
+ * all vision-capable so they accept the receipt/invoice image and PDF inputs.
+ * Pricing is the standard per-MTok list price from each provider's public
+ * pricing page as of 2026-07 (sourced inline). Published cost numbers use the
+ * durable list price, not promotional rates, so they don't expire.
+ */
+const CATALOG: Record<Provider, ProviderSpec> = {
+  // platform.claude.com/docs/en/pricing
+  anthropic: {
+    apiKeyEnv: 'ANTHROPIC_API_KEY',
+    create: (id) => anthropic(id),
+    models: [
+      { name: 'claude-opus-4-8', pricing: { inputPerMTokUSD: 5, outputPerMTokUSD: 25 } },
+      { name: 'claude-sonnet-5', pricing: { inputPerMTokUSD: 3, outputPerMTokUSD: 15 } },
+      { name: 'claude-haiku-4-5', pricing: { inputPerMTokUSD: 1, outputPerMTokUSD: 5 } },
+    ],
+  },
+  // developers.openai.com/api/docs/pricing
+  openai: {
+    apiKeyEnv: 'OPENAI_API_KEY',
+    create: (id) => openai(id),
+    models: [
+      { name: 'gpt-5.6-sol', pricing: { inputPerMTokUSD: 5, outputPerMTokUSD: 30 } },
+      { name: 'gpt-5.6-luna', pricing: { inputPerMTokUSD: 1, outputPerMTokUSD: 6 } },
+      { name: 'gpt-5.4-mini', pricing: { inputPerMTokUSD: 0.75, outputPerMTokUSD: 4.5 } },
+    ],
+  },
+  // ai.google.dev/gemini-api/docs/pricing (standard rate, <=200k-token context)
+  google: {
+    apiKeyEnv: 'GOOGLE_GENERATIVE_AI_API_KEY',
+    create: (id) => google(id),
+    models: [
+      { name: 'gemini-2.5-pro', pricing: { inputPerMTokUSD: 1.25, outputPerMTokUSD: 10 } },
+      { name: 'gemini-2.5-flash', pricing: { inputPerMTokUSD: 0.3, outputPerMTokUSD: 2.5 } },
+      { name: 'gemini-2.5-flash-lite', pricing: { inputPerMTokUSD: 0.1, outputPerMTokUSD: 0.4 } },
+    ],
+  },
+};
+
+function parseProviders(raw: string): Provider[] {
+  const names = raw.split(/[,\s]+/).filter((s) => s.length > 0);
+  const selected: Provider[] = [];
+  for (const name of names) {
+    if (!PROVIDERS.includes(name as Provider)) {
+      throw new Error(`Unknown provider "${name}" in EVAL_PROVIDERS; valid values are ${PROVIDERS.join(', ')}.`);
+    }
+    if (!selected.includes(name as Provider)) selected.push(name as Provider);
+  }
+  if (selected.length === 0) {
+    throw new Error(`EVAL_PROVIDERS is empty; set it to a comma-separated subset of ${PROVIDERS.join(', ')}.`);
+  }
+  return selected;
+}
+
+/**
+ * Which providers to benchmark. `EVAL_PROVIDERS` selects them explicitly
+ * (comma-separated, e.g. `openai` or `openai,google`); each named provider
+ * must have its API key set. When unset, every provider whose API key is
+ * present runs — so setting only `OPENAI_API_KEY` benchmarks OpenAI alone.
+ */
+export function selectProviders(env: NodeJS.ProcessEnv = process.env): Provider[] {
+  const raw = env['EVAL_PROVIDERS'];
+  if (raw !== undefined) {
+    const requested = parseProviders(raw);
+    const missing = requested.filter((p) => env[CATALOG[p].apiKeyEnv] === undefined);
+    if (missing.length > 0) {
+      const detail = missing.map((p) => `${p} (${CATALOG[p].apiKeyEnv})`).join(', ');
+      throw new Error(`Missing API key for selected provider(s): ${detail}.`);
+    }
+    return requested;
+  }
+  const available = PROVIDERS.filter((p) => env[CATALOG[p].apiKeyEnv] !== undefined);
+  if (available.length === 0) {
+    const keys = PROVIDERS.map((p) => CATALOG[p].apiKeyEnv).join(', ');
+    throw new Error(`No provider API key found; set one of ${keys}, or pick providers with EVAL_PROVIDERS.`);
+  }
+  return available;
+}
+
+/** The models to run this benchmark, resolved from the selected providers. */
+export function benchmarkModels(env: NodeJS.ProcessEnv = process.env): EvalModel[] {
+  return selectProviders(env).flatMap((provider) => {
+    const spec = CATALOG[provider];
+    return spec.models.map((m) => ({ name: m.name, model: spec.create(m.name), pricing: m.pricing }));
+  });
+}
diff --git a/packages/evals/src/normalize.ts b/packages/evals/src/normalize.ts
new file mode 100644
index 0000000..6862a22
--- /dev/null
+++ b/packages/evals/src/normalize.ts
@@ -0,0 +1,46 @@
+import type { CompareKind } from './types.js';
+
+/** NFKC, case-fold, collapse whitespace. */
+export function normalizeText(value: string): string {
+  return value.normalize('NFKC').toLowerCase().replace(/\s+/g, ' ').trim();
+}
+
+/**
+ * Reduce a printed amount to sign + digits: "Rp 24,000" → "24000",
+ * "-60.000" → "-60000", "@11000" → "11000". Locale-safe on the benchmark
+ * documents because their amounts never carry meaningful decimal fractions
+ * with the same separator as the thousands separator.
+ */
+export function normalizeMoney(value: string): string {
+  const digits = value.replace(/[^0-9-]/g, '');
+  const negative = digits.startsWith('-') || /\(\s*[0-9.,]+\s*\)/.test(value);
+  const bare = digits.replace(/-/g, '');
+  if (bare === '') return '';
+  const trimmed = bare.replace(/^0+(?=\d)/, '');
+  return negative ? `-${trimmed}` : trimmed;
+}
+
+/** Lenient count parse: "2.00" → "2", "1X" → "1", "1.5" → "1.5". */
+export function normalizeCount(value: string): string {
+  const match = value.replace(/,/g, '.').match(/-?\d+(?:\.\d+)?/);
+  if (!match) return normalizeText(value);
+  const num = Number.parseFloat(match[0]);
+  return Number.isFinite(num) ? String(num) : normalizeText(value);
+}
+
+export function normalizeValue(value: string, compare: CompareKind): string {
+  switch (compare) {
+    case 'text':
+      return normalizeText(value);
+    case 'money':
+      return normalizeMoney(value);
+    case 'count':
+      return normalizeCount(value);
+  }
+}
+
+/** Null matches only null; otherwise compare under the field's normalizer. */
+export function valuesMatch(expected: string | null, predicted: string | null, compare: CompareKind): boolean {
+  if (expected === null || predicted === null) return expected === predicted;
+  return normalizeValue(expected, compare) === normalizeValue(predicted, compare);
+}
diff --git a/packages/evals/src/report.ts b/packages/evals/src/report.ts
new file mode 100644
index 0000000..42a361e
--- /dev/null
+++ b/packages/evals/src/report.ts
@@ -0,0 +1,128 @@
+import { summarizeRun, type ModelSummary, type SchemaSummary } from './metrics.js';
+import type { RunRecord, SchemaId } from './types.js';
+
+const SCHEMA_LABELS: Record<SchemaId, string> = {
+  receipt: 'Receipts — CORD-v2 (photographed shop receipts)',
+  invoice: 'Invoices — DocILE (real business documents, PDF)',
+};
+
+function pct(value: number | null): string {
+  return value === null ? '—' : `${(value * 100).toFixed(1)}%`;
+}
+
+function usd(value: number | null): string {
+  return value === null ? '—' : `$${value.toFixed(2)}`;
+}
+
+function mainTable(summaries: ModelSummary[], schema: SchemaId): string | null {
+  const rows = summaries
+    .map((s) => ({ model: s.model, sum: s.perSchema[schema] }))
+    .filter((r): r is { model: string; sum: SchemaSummary } => r.sum !== undefined);
+  if (rows.length === 0) return null;
+  const lines = [
+    '| Model | Docs | Field accuracy | Grounding hit@0.5 | Mean IoU | Cost / 1k docs |',
+    '|---|---|---|---|---|---|',
+    ...rows.map(
+      ({ model, sum }) =>
+        `| ${model} | ${sum.docs}${sum.failedDocs > 0 ? ` (${sum.failedDocs} failed)` : ''} | ${pct(
+          sum.fieldAccuracy,
+        )} | ${pct(sum.grounding.hitAt50)} | ${pct(sum.grounding.meanIoU)} | ${usd(sum.cost.per1kDocsUSD)} |`,
+    ),
+  ];
+  return lines.join('\n');
+}
+
+function perFieldTable(summaries: ModelSummary[], schema: SchemaId): string | null {
+  const rows = summaries
+    .map((s) => ({ model: s.model, sum: s.perSchema[schema] }))
+    .filter((r): r is { model: string; sum: SchemaSummary } => r.sum !== undefined);
+  if (rows.length === 0) return null;
+  const fields = rows[0]?.sum.perField.map((f) => f.field) ?? [];
+  const lines = [
+    `| Field | ${rows.map((r) => r.model).join(' | ')} |`,
+    `|---|${rows.map(() => '---').join('|')}|`,
+    ...fields.map((field) => {
+      const cells = rows.map(({ sum }) => {
+        const entry = sum.perField.find((f) => f.field === field);
+        return entry === undefined ? '—' : `${pct(entry.accuracy)} (${entry.count})`;
+      });
+      return `| \`${field}\` | ${cells.join(' | ')} |`;
+    }),
+  ];
+  return lines.join('\n');
+}
+
+/** The summary table embedded in the README, one section per document type. */
+export function renderReadmeTable(record: RunRecord): string {
+  const summaries = record.runs.map(summarizeRun);
+  const sections: string[] = [];
+  for (const schema of ['invoice', 'receipt'] as const) {
+    const table = mainTable(summaries, schema);
+    if (table !== null) sections.push(`**${SCHEMA_LABELS[schema]}**\n\n${table}`);
+  }
+  return sections.join('\n\n');
+}
+
+/** The full benchmark page (docs/benchmark.md). */
+export function renderBenchmarkPage(record: RunRecord): string {
+  const summaries = record.runs.map(summarizeRun);
+  const parts: string[] = [
+    '# extractkit benchmark',
+    '',
+    'Field-level extraction accuracy, grounding accuracy, and cost, measured on pinned public',
+    'documents: receipts from [CORD-v2](https://huggingface.co/datasets/naver-clova-ix/cord-v2)',
+    '(NAVER CLOVA, CC BY 4.0) and invoices from [DocILE](https://docile.rossum.ai/) (Rossum).',
+    'Documents are pinned by id + checksum in',
+    '[`packages/evals/data/manifest.json`](../packages/evals/data/manifest.json) and fetched from',
+    'their canonical hosts at eval time — see [`packages/evals`](../packages/evals) to reproduce.',
+    '',
+    `Run started ${record.startedAt}. Manifest checksum \`${record.manifestChecksum.slice(0, 12)}\`.`,
+    '',
+    '**Metrics.** *Field accuracy*: fraction of ground-truth fields whose predicted value matches',
+    'under normalized comparison (whitespace/case for text, digits-and-sign for amounts, lenient',
+    'numeric parse for quantities); a field the document does not carry counts as correct only when',
+    'the model returned null for it. *Grounding hit@0.5*: among fields with a correct value and a',
+    'ground-truth region, the fraction where the predicted bounding box overlaps an annotated region',
+    'with IoU ≥ 0.5 on the right page; a missing bbox or wrong page scores 0. *Cost / 1k docs*:',
+    'measured token usage priced at published per-token rates.',
+    '',
+  ];
+
+  for (const schema of ['invoice', 'receipt'] as const) {
+    const table = mainTable(summaries, schema);
+    if (table === null) continue;
+    parts.push(`## ${SCHEMA_LABELS[schema]}`, '', table, '');
+    const fieldTable = perFieldTable(summaries, schema);
+    if (fieldTable !== null) parts.push('### Accuracy per field', '', fieldTable, '');
+    const extras = summaries
+      .map((s) => ({ model: s.model, sum: s.perSchema[schema] }))
+      .filter((r): r is { model: string; sum: SchemaSummary } => r.sum !== undefined)
+      .filter((r) => r.sum.extraLineItems > 0);
+    if (extras.length > 0) {
+      parts.push(
+        'Hallucinated line items (predicted beyond ground truth, not counted in field accuracy): ' +
+          extras.map((r) => `${r.model}: ${r.sum.extraLineItems}`).join(', ') +
+          '.',
+        '',
+      );
+    }
+  }
+
+  parts.push(
+    '## Caveats',
+    '',
+    '- Both datasets are public and widely cited; frontier models have likely seen them in training.',
+    '  Read these numbers as a comparative measurement across models under identical conditions, not',
+    '  an absolute capability claim.',
+    '- CORD receipts are Indonesian (Latin script); the numbers are not universal across locales.',
+    '- Line items are aligned to ground truth by printed order; a correct item at the wrong position',
+    '  scores as wrong.',
+    '',
+    'Receipt data © NAVER CLOVA, [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/), via the',
+    'official [cord-v2](https://huggingface.co/datasets/naver-clova-ix/cord-v2) dataset. DocILE',
+    'documents are not redistributed; runners fetch them with their own access token.',
+    '',
+  );
+
+  return parts.join('\n');
+}
diff --git a/packages/evals/src/runner.ts b/packages/evals/src/runner.ts
new file mode 100644
index 0000000..10cea3b
--- /dev/null
+++ b/packages/evals/src/runner.ts
@@ -0,0 +1,62 @@
+import { extract, ExtractKitError, MissingRequiredFieldsError } from 'extractkit';
+import type { ExtractUsage } from 'extractkit';
+import { scoreExtraction, scoreFailure } from './metrics.js';
+import { schemas } from './schemas.js';
+import type { DocResult, EvalDocument, EvalModel, ModelRun } from './types.js';
+
+export interface RunOptions {
+  /** Concurrent extractions per model. Default 4. */
+  concurrency?: number;
+  onDocDone?: (result: DocResult) => void;
+}
+
+function toUsage(usage: ExtractUsage): DocResult['usage'] {
+  return {
+    inputTokens: usage.inputTokens,
+    outputTokens: usage.outputTokens,
+    modelCalls: usage.modelCalls,
+    costUSD: usage.costUSD,
+  };
+}
+
+async function runDoc(model: EvalModel, doc: EvalDocument): Promise<DocResult> {
+  const base = { docId: doc.id, dataset: doc.dataset, schema: doc.schema };
+  try {
+    const result = await extract({
+      schema: schemas[doc.schema],
+      document: { data: doc.bytes, mediaType: doc.mediaType },
+      model: model.model,
+      temperature: 0,
+      ...(model.pricing !== undefined ? { pricing: model.pricing } : {}),
+    });
+    return { ...base, error: null, ...scoreExtraction(doc, result), usage: toUsage(result.usage) };
+  } catch (err) {
+    const message = err instanceof Error ? `${err.name}: ${err.message}` : String(err);
+    if (err instanceof MissingRequiredFieldsError) {
+      return { ...base, error: message, ...scoreExtraction(doc, err.partial), usage: toUsage(err.usage) };
+    }
+    const usage = err instanceof ExtractKitError && 'usage' in err ? toUsage(err.usage as ExtractUsage) : null;
+    return { ...base, error: message, ...scoreFailure(doc), usage };
+  }
+}
+
+/** Run one model over the benchmark documents with bounded concurrency.
+ * Document order in the result matches the input order. */
+export async function runModel(model: EvalModel, docs: EvalDocument[], options: RunOptions = {}): Promise<ModelRun> {
+  const concurrency = options.concurrency ?? 4;
+  const results: DocResult[] = new Array(docs.length);
+  let next = 0;
+
+  async function worker(): Promise<void> {
+    while (next < docs.length) {
+      const index = next++;
+      const doc = docs[index] as EvalDocument;
+      const result = await runDoc(model, doc);
+      results[index] = result;
+      options.onDocDone?.(result);
+    }
+  }
+
+  await Promise.all(Array.from({ length: Math.min(concurrency, docs.length) }, worker));
+  return { model: model.name, docs: results };
+}
diff --git a/packages/evals/src/schemas.ts b/packages/evals/src/schemas.ts
new file mode 100644
index 0000000..10ca4c1
--- /dev/null
+++ b/packages/evals/src/schemas.ts
@@ -0,0 +1,60 @@
+import { z } from 'zod';
+
+/**
+ * Receipt schema for the CORD-v2 half of the benchmark. Amount fields are
+ * strings extracted as printed (CORD receipts mix "." and "," thousands
+ * separators, so numeric parsing would test locale guessing, not extraction).
+ * Only fields with CORD ground truth appear here — CORD does not annotate
+ * merchant name or date.
+ */
+export const receiptSchema = z.object({
+  lineItems: z
+    .array(
+      z.object({
+        description: z.string().describe('Item name as printed, e.g. "ICE BLACKCOFFE"'),
+        quantity: z.string().nullable().describe('Quantity as printed, e.g. "2" or "1X"'),
+        unitPrice: z.string().nullable().describe('Per-unit price as printed, e.g. "@11000"'),
+        amount: z.string().nullable().describe('Line total as printed, e.g. "24,000"'),
+      }),
+    )
+    .describe('Every purchased item in printed order, including sub-items listed under another item'),
+  subtotal: z.string().nullable().describe('Subtotal before tax/discount/service, as printed'),
+  discount: z.string().nullable().describe('Discount amount as printed, e.g. "-60.000" or "19,400"'),
+  serviceCharge: z.string().nullable().describe('Service charge amount as printed'),
+  tax: z.string().nullable().describe('Tax amount as printed'),
+  total: z.string().describe('Final charged total as printed'),
+});
+
+export type Receipt = z.output<typeof receiptSchema>;
+
+/**
+ * Invoice schema for the DocILE half of the benchmark. Amounts are strings
+ * as printed for the same reason as receipts.
+ */
+export const invoiceSchema = z.object({
+  vendorName: z.string().nullable().describe('Name of the party issuing the invoice'),
+  invoiceNumber: z.string().nullable().describe('Invoice identifier assigned by the vendor'),
+  issueDate: z.string().nullable().describe('Date the invoice was issued, as printed'),
+  dueDate: z.string().nullable().describe('Payment due date, as printed'),
+  currency: z.string().nullable().describe('Currency code or symbol used for the amounts'),
+  subtotal: z.string().nullable().describe('Total before tax, as printed'),
+  tax: z.string().nullable().describe('Total tax amount, as printed'),
+  total: z.string().nullable().describe('Total amount due, as printed'),
+  lineItems: z
+    .array(
+      z.object({
+        description: z.string().nullable().describe('Line item description as printed'),
+        quantity: z.string().nullable().describe('Quantity as printed'),
+        unitPrice: z.string().nullable().describe('Per-unit price as printed'),
+        amount: z.string().nullable().describe('Line total as printed'),
+      }),
+    )
+    .describe('Every line item in printed order'),
+});
+
+export type Invoice = z.output<typeof invoiceSchema>;
+
+export const schemas = {
+  receipt: receiptSchema,
+  invoice: invoiceSchema,
+} as const;
diff --git a/packages/evals/src/types.ts b/packages/evals/src/types.ts
new file mode 100644
index 0000000..bdd2089
--- /dev/null
+++ b/packages/evals/src/types.ts
@@ -0,0 +1,107 @@
+import type { BBox, Pricing, SupportedMediaType } from 'extractkit';
+import type { LanguageModel } from 'ai';
+
+export type DatasetId = 'cord' | 'docile';
+export type SchemaId = 'receipt' | 'invoice';
+
+/**
+ * How a field's predicted value is compared to ground truth:
+ * - `text`: Unicode-normalized, case-folded, whitespace-collapsed equality.
+ * - `money`: digits-and-sign equality, ignoring currency symbols and
+ *   separators ("24,000" ≡ "24.000" ≡ "24000"). Robust to locale-ambiguous
+ *   thousands separators on the benchmark receipts.
+ * - `count`: lenient numeric parse ("2.00" ≡ "2", "1X" ≡ "1").
+ */
+export type CompareKind = 'text' | 'money' | 'count';
+
+/** One annotated occurrence of a value: 0-based page + normalized (0–1) box. */
+export interface GroundTruthRegion {
+  page: number;
+  bbox: BBox;
+}
+
+/** One expected leaf value for a document. `value: null` means the schema
+ * field exists but the document does not carry it. */
+export interface GroundTruthField {
+  /** Dot path into the extraction schema, e.g. `lineItems.0.description`. */
+  path: string;
+  value: string | null;
+  /** Other acceptable renderings when the value is printed more than once
+   * with different text (e.g. two date formats). */
+  altValues?: string[];
+  compare: CompareKind;
+  /** Every annotated region carrying the value; a predicted bbox is scored
+   * against its best match. Empty when `value` is null or no box exists. */
+  regions: GroundTruthRegion[];
+}
+
+export interface EvalDocument {
+  /** Stable benchmark id, e.g. `cord/test/17`. */
+  id: string;
+  dataset: DatasetId;
+  schema: SchemaId;
+  bytes: Uint8Array;
+  mediaType: SupportedMediaType;
+  pages: number;
+  fields: GroundTruthField[];
+  /** Ground-truth line-item count, used to detect hallucinated items. */
+  lineItemCount: number;
+}
+
+export interface EvalModel {
+  /** Display name used in reports, e.g. `claude-sonnet-5`. */
+  name: string;
+  model: LanguageModel;
+  pricing?: Pricing;
+}
+
+/** Per-field outcome of one extraction. */
+export interface FieldResult {
+  path: string;
+  compare: CompareKind;
+  expected: string | null;
+  /** Predicted leaf value rendered to a string; null for extracted-null. */
+  predicted: string | null;
+  valueCorrect: boolean;
+  /**
+   * Grounding score: best IoU between the predicted bbox and the
+   * ground-truth regions on the predicted page. 0 when the model gave no
+   * bbox or the wrong page; null when not scoreable (value wrong, or no
+   * ground-truth region for this field).
+   */
+  iou: number | null;
+  predictedPage: number | null;
+}
+
+export interface DocResult {
+  docId: string;
+  dataset: DatasetId;
+  schema: SchemaId;
+  /**
+   * Set when extraction threw. Fields are still scored against the partial
+   * extraction when the error carried one (missing required fields);
+   * otherwise every field counts as incorrect.
+   */
+  error: string | null;
+  fields: FieldResult[];
+  /** Predicted line items beyond the ground-truth count. */
+  extraLineItems: number;
+  usage: {
+    inputTokens: number;
+    outputTokens: number;
+    modelCalls: number;
+    costUSD: number | null;
+  } | null;
+}
+
+export interface ModelRun {
+  model: string;
+  docs: DocResult[];
+}
+
+/** A completed benchmark run, serialized to results/<timestamp>.json. */
+export interface RunRecord {
+  startedAt: string;
+  manifestChecksum: string;
+  runs: ModelRun[];
+}
diff --git a/packages/evals/test/cord.test.ts b/packages/evals/test/cord.test.ts
new file mode 100644
index 0000000..ed3a808
--- /dev/null
+++ b/packages/evals/test/cord.test.ts
@@ -0,0 +1,181 @@
+import { describe, expect, it } from 'vitest';
+import { cordToGroundTruth, CordMappingError, type CordGroundTruth } from '../src/datasets/cord.js';
+
+function quad(x0: number, y0: number, x1: number, y1: number) {
+  return { x1: x0, y1: y0, x2: x1, y2: y0, x3: x1, y3: y1, x4: x0, y4: y1 };
+}
+
+function line(category: string, groupId: number, text: string, box: [number, number, number, number], subGroupId?: number) {
+  return {
+    category,
+    group_id: groupId,
+    ...(subGroupId !== undefined ? { sub_group_id: subGroupId } : {}),
+    words: [{ text, quad: quad(...box) }],
+  };
+}
+
+const META = { image_id: 0, image_size: { width: 100, height: 200 } };
+
+describe('cordToGroundTruth', () => {
+  it('maps a single-item receipt with scalars', () => {
+    const gt: CordGroundTruth = {
+      meta: META,
+      gt_parse: {
+        menu: { nm: 'TICKET CP', cnt: '2', price: '60.000' },
+        sub_total: { subtotal_price: '60.000', tax_price: '5.455' },
+        total: { total_price: '65.455' },
+      },
+      valid_line: [
+        line('menu.nm', 1, 'TICKET CP', [10, 20, 50, 30]),
+        line('menu.cnt', 1, '2', [5, 20, 8, 30]),
+        line('menu.price', 1, '60.000', [60, 20, 90, 30]),
+        line('sub_total.subtotal_price', 2, '60.000', [60, 40, 90, 50]),
+        line('sub_total.tax_price', 2, '5.455', [60, 52, 90, 60]),
+        line('total.total_price', 3, '65.455', [60, 64, 90, 72]),
+      ],
+    };
+    const { fields, lineItemCount } = cordToGroundTruth(gt);
+    expect(lineItemCount).toBe(1);
+
+    const byPath = new Map(fields.map((f) => [f.path, f]));
+    expect(byPath.get('lineItems.0.description')?.value).toBe('TICKET CP');
+    expect(byPath.get('lineItems.0.quantity')?.value).toBe('2');
+    expect(byPath.get('lineItems.0.unitPrice')?.value).toBeNull();
+    expect(byPath.get('lineItems.0.amount')?.value).toBe('60.000');
+    expect(byPath.get('subtotal')?.value).toBe('60.000');
+    expect(byPath.get('tax')?.value).toBe('5.455');
+    expect(byPath.get('discount')?.value).toBeNull();
+    expect(byPath.get('serviceCharge')?.value).toBeNull();
+    expect(byPath.get('total')?.value).toBe('65.455');
+
+    // Boxes are normalized by image size, page 0.
+    const nm = byPath.get('lineItems.0.description');
+    expect(nm?.regions).toEqual([{ page: 0, bbox: { x0: 0.1, y0: 0.1, x1: 0.5, y1: 0.15 } }]);
+    expect(byPath.get('lineItems.0.unitPrice')?.regions).toEqual([]);
+  });
+
+  it('maps an array menu in ascending group order', () => {
+    const gt: CordGroundTruth = {
+      meta: META,
+      gt_parse: {
+        menu: [
+          { nm: 'FIRST', price: '1.000' },
+          { nm: 'SECOND', price: '2.000' },
+        ],
+        total: { total_price: '3.000' },
+      },
+      valid_line: [
+        // Deliberately unsorted input order.
+        line('menu.nm', 7, 'SECOND', [10, 40, 50, 50]),
+        line('menu.nm', 3, 'FIRST', [10, 20, 50, 30]),
+        line('menu.price', 3, '1.000', [60, 20, 90, 30]),
+        line('menu.price', 7, '2.000', [60, 40, 90, 50]),
+        line('total.total_price', 9, '3.000', [60, 60, 90, 70]),
+      ],
+    };
+    const { fields, lineItemCount } = cordToGroundTruth(gt);
+    expect(lineItemCount).toBe(2);
+    const byPath = new Map(fields.map((f) => [f.path, f]));
+    expect(byPath.get('lineItems.0.description')?.value).toBe('FIRST');
+    expect(byPath.get('lineItems.1.description')?.value).toBe('SECOND');
+    expect(byPath.get('lineItems.1.amount')?.regions[0]?.bbox.y0).toBeCloseTo(0.2);
+  });
+
+  it('flattens sub-items after their parent', () => {
+    const gt: CordGroundTruth = {
+      meta: META,
+      gt_parse: {
+        menu: { nm: 'JASMINE MT', cnt: '1', price: '24,000', sub: { nm: 'COCONUT JELLY', price: '4,000' } },
+        total: { total_price: '28,000' },
+      },
+      valid_line: [
+        line('menu.nm', 1, 'JASMINE MT', [10, 20, 50, 30]),
+        line('menu.cnt', 1, '1', [5, 20, 8, 30]),
+        line('menu.price', 1, '24,000', [60, 20, 90, 30]),
+        line('menu.sub.nm', 1, 'COCONUT JELLY', [15, 32, 55, 40], 0),
+        line('menu.sub.price', 1, '4,000', [60, 32, 90, 40], 0),
+        line('total.total_price', 2, '28,000', [60, 60, 90, 70]),
+      ],
+    };
+    const { fields, lineItemCount } = cordToGroundTruth(gt);
+    expect(lineItemCount).toBe(2);
+    const byPath = new Map(fields.map((f) => [f.path, f]));
+    expect(byPath.get('lineItems.0.description')?.value).toBe('JASMINE MT');
+    expect(byPath.get('lineItems.1.description')?.value).toBe('COCONUT JELLY');
+    expect(byPath.get('lineItems.1.amount')?.value).toBe('4,000');
+    expect(byPath.get('lineItems.1.amount')?.regions).toHaveLength(1);
+  });
+
+  it('joins multi-line values into one bbox union', () => {
+    const gt: CordGroundTruth = {
+      meta: META,
+      gt_parse: { menu: { nm: 'VERY LONG ITEM NAME' }, total: { total_price: '1.000' } },
+      valid_line: [
+        line('menu.nm', 1, 'VERY LONG', [10, 20, 50, 30]),
+        line('menu.nm', 1, 'ITEM NAME', [10, 32, 45, 42]),
+        line('total.total_price', 2, '1.000', [60, 60, 90, 70]),
+      ],
+    };
+    const { fields } = cordToGroundTruth(gt);
+    const nm = fields.find((f) => f.path === 'lineItems.0.description');
+    expect(nm?.regions[0]?.bbox).toEqual({ x0: 0.1, y0: 0.1, x1: 0.5, y1: 0.21 });
+  });
+
+  it('grounds a labeled scalar on the value words only', () => {
+    const gt: CordGroundTruth = {
+      meta: META,
+      gt_parse: { menu: { nm: 'A' }, sub_total: { tax_price: '3,650' }, total: { total_price: '40,150' } },
+      valid_line: [
+        line('menu.nm', 1, 'A', [10, 20, 50, 30]),
+        {
+          category: 'sub_total.tax_price',
+          group_id: 2,
+          words: [
+            { text: 'PB1', quad: quad(10, 40, 25, 50) },
+            { text: '10%', quad: quad(30, 40, 45, 50) },
+            { text: '3,650', quad: quad(60, 40, 90, 50) },
+          ],
+        },
+        line('total.total_price', 3, '40,150', [60, 60, 90, 70]),
+      ],
+    };
+    const { fields } = cordToGroundTruth(gt);
+    const tax = fields.find((f) => f.path === 'tax');
+    expect(tax?.value).toBe('3,650');
+    // Only the value word grounds the field, not the "PB1 10%" label.
+    expect(tax?.regions).toEqual([{ page: 0, bbox: { x0: 0.6, y0: 0.2, x1: 0.9, y1: 0.25 } }]);
+  });
+
+  it('rejects a document whose annotated text disagrees with gt_parse', () => {
+    const gt: CordGroundTruth = {
+      meta: META,
+      gt_parse: { menu: { nm: 'EXPECTED' }, total: { total_price: '1.000' } },
+      valid_line: [
+        line('menu.nm', 1, 'DIFFERENT', [10, 20, 50, 30]),
+        line('total.total_price', 2, '1.000', [60, 60, 90, 70]),
+      ],
+    };
+    expect(() => cordToGroundTruth(gt)).toThrow(CordMappingError);
+  });
+
+  it('rejects a document whose item count disagrees with the annotation groups', () => {
+    const gt: CordGroundTruth = {
+      meta: META,
+      gt_parse: { menu: [{ nm: 'A' }, { nm: 'B' }], total: { total_price: '1.000' } },
+      valid_line: [
+        line('menu.nm', 1, 'A', [10, 20, 50, 30]),
+        line('total.total_price', 2, '1.000', [60, 60, 90, 70]),
+      ],
+    };
+    expect(() => cordToGroundTruth(gt)).toThrow(CordMappingError);
+  });
+
+  it('rejects non-string scalar values', () => {
+    const gt: CordGroundTruth = {
+      meta: META,
+      gt_parse: { menu: { nm: 'A' }, total: { total_price: ['1.000', '2.000'] } },
+      valid_line: [line('menu.nm', 1, 'A', [10, 20, 50, 30])],
+    };
+    expect(() => cordToGroundTruth(gt)).toThrow(CordMappingError);
+  });
+});
diff --git a/packages/evals/test/docile.test.ts b/packages/evals/test/docile.test.ts
new file mode 100644
index 0000000..0081ab4
--- /dev/null
+++ b/packages/evals/test/docile.test.ts
@@ -0,0 +1,120 @@
+import { describe, expect, it } from 'vitest';
+import { docileToGroundTruth, type DocileAnnotation, type DocileField } from '../src/datasets/docile.js';
+
+function field(
+  fieldtype: string,
+  text: string,
+  over: Partial<DocileField> = {},
+): DocileField {
+  return { fieldtype, text, page: 0, bbox: [0.1, 0.1, 0.3, 0.12], ...over };
+}
+
+function annotation(over: Partial<DocileAnnotation> = {}): DocileAnnotation {
+  return {
+    field_extractions: [],
+    line_item_extractions: [],
+    metadata: { page_count: 1, document_type: 'tax_invoice', currency: 'USD' },
+    ...over,
+  };
+}
+
+describe('docileToGroundTruth', () => {
+  it('maps header fieldtypes onto the invoice schema', () => {
+    const { fields } = docileToGroundTruth(
+      annotation({
+        field_extractions: [
+          field('vendor_name', 'ACME Corp'),
+          field('document_id', 'INV-99'),
+          field('date_issue', '01/15/2021'),
+          field('amount_total_net', '100.00'),
+          field('amount_total_tax', '8.00'),
+          field('amount_total_gross', '108.00'),
+        ],
+      }),
+    );
+    const byPath = new Map(fields.map((f) => [f.path, f]));
+    expect(byPath.get('vendorName')?.value).toBe('ACME Corp');
+    expect(byPath.get('invoiceNumber')?.value).toBe('INV-99');
+    expect(byPath.get('issueDate')?.value).toBe('01/15/2021');
+    expect(byPath.get('dueDate')?.value).toBeNull();
+    expect(byPath.get('subtotal')?.value).toBe('100.00');
+    expect(byPath.get('tax')?.value).toBe('8.00');
+    expect(byPath.get('total')?.value).toBe('108.00');
+    expect(byPath.get('vendorName')?.regions).toHaveLength(1);
+    expect(byPath.get('dueDate')?.regions).toEqual([]);
+  });
+
+  it('collects repeated occurrences: all regions, differing texts as altValues', () => {
+    const { fields } = docileToGroundTruth(
+      annotation({
+        field_extractions: [
+          field('document_id', 'INV-99', { page: 1, bbox: [0.7, 0.05, 0.9, 0.07] }),
+          field('document_id', 'INV-99', { page: 0, bbox: [0.7, 0.05, 0.9, 0.07] }),
+          field('date_issue', 'Jan 15, 2021', { page: 0, bbox: [0.1, 0.3, 0.3, 0.32] }),
+          field('date_issue', '01/15/2021', { page: 0, bbox: [0.1, 0.1, 0.3, 0.12] }),
+        ],
+      }),
+    );
+    const byPath = new Map(fields.map((f) => [f.path, f]));
+    const invoiceNumber = byPath.get('invoiceNumber');
+    // Same text on both pages: one canonical value, two acceptable regions.
+    expect(invoiceNumber?.value).toBe('INV-99');
+    expect(invoiceNumber?.altValues).toBeUndefined();
+    expect(invoiceNumber?.regions).toHaveLength(2);
+    expect(invoiceNumber?.regions[0]?.page).toBe(0);
+    // Different rendering: reading-order first is canonical, other is an alt.
+    const issueDate = byPath.get('issueDate');
+    expect(issueDate?.value).toBe('01/15/2021');
+    expect(issueDate?.altValues).toEqual(['Jan 15, 2021']);
+  });
+
+  it('orders line items by reading order of their topmost field', () => {
+    const { fields, lineItemCount } = docileToGroundTruth(
+      annotation({
+        line_item_extractions: [
+          field('line_item_description', 'Second item', { line_item_id: 5, bbox: [0.1, 0.5, 0.4, 0.52] }),
+          field('line_item_description', 'First item', { line_item_id: 9, bbox: [0.1, 0.3, 0.4, 0.32] }),
+          field('line_item_amount_net', '10.00', { line_item_id: 9, bbox: [0.7, 0.3, 0.9, 0.32] }),
+          field('line_item_quantity', '2', { line_item_id: 5, bbox: [0.5, 0.5, 0.6, 0.52] }),
+        ],
+      }),
+    );
+    expect(lineItemCount).toBe(2);
+    const byPath = new Map(fields.map((f) => [f.path, f]));
+    expect(byPath.get('lineItems.0.description')?.value).toBe('First item');
+    expect(byPath.get('lineItems.0.amount')?.value).toBe('10.00');
+    expect(byPath.get('lineItems.1.description')?.value).toBe('Second item');
+    expect(byPath.get('lineItems.1.quantity')?.value).toBe('2');
+    expect(byPath.get('lineItems.1.amount')?.value).toBeNull();
+  });
+
+  it('prefers net over gross for unit price and amount', () => {
+    const { fields } = docileToGroundTruth(
+      annotation({
+        line_item_extractions: [
+          field('line_item_description', 'Widget', { line_item_id: 1 }),
+          field('line_item_unit_price_gross', '12.00', { line_item_id: 1, bbox: [0.5, 0.1, 0.6, 0.12] }),
+          field('line_item_amount_net', '10.00', { line_item_id: 1, bbox: [0.7, 0.1, 0.9, 0.12] }),
+        ],
+      }),
+    );
+    const byPath = new Map(fields.map((f) => [f.path, f]));
+    // Only gross unit price exists → fall back to it.
+    expect(byPath.get('lineItems.0.unitPrice')?.value).toBe('12.00');
+    expect(byPath.get('lineItems.0.amount')?.value).toBe('10.00');
+  });
+
+  it('joins multi-fragment line-item values in reading order', () => {
+    const { fields } = docileToGroundTruth(
+      annotation({
+        line_item_extractions: [
+          field('line_item_description', 'continued on line two', { line_item_id: 1, bbox: [0.1, 0.32, 0.4, 0.34] }),
+          field('line_item_description', 'A long description', { line_item_id: 1, bbox: [0.1, 0.3, 0.4, 0.32] }),
+        ],
+      }),
+    );
+    const description = fields.find((f) => f.path === 'lineItems.0.description');
+    expect(description?.value).toBe('A long description continued on line two');
+    expect(description?.regions).toHaveLength(2);
+  });
+});
diff --git a/packages/evals/test/metrics.test.ts b/packages/evals/test/metrics.test.ts
new file mode 100644
index 0000000..494d4a0
--- /dev/null
+++ b/packages/evals/test/metrics.test.ts
@@ -0,0 +1,178 @@
+import type { ExtractedField } from 'extractkit';
+import { describe, expect, it } from 'vitest';
+import { iou, scoreExtraction, scoreFailure, summarizeRun } from '../src/metrics.js';
+import type { EvalDocument, GroundTruthField, ModelRun } from '../src/types.js';
+
+function leaf(value: unknown, over: Partial<ExtractedField<unknown>> = {}): ExtractedField<unknown> {
+  return { value, confidence: 0.9, page: 0, bbox: { x0: 0.1, y0: 0.1, x1: 0.5, y1: 0.2 }, ...over };
+}
+
+function gtField(over: Partial<GroundTruthField> & Pick<GroundTruthField, 'path' | 'value'>): GroundTruthField {
+  return {
+    compare: 'text',
+    regions: over.value === null ? [] : [{ page: 0, bbox: { x0: 0.1, y0: 0.1, x1: 0.5, y1: 0.2 } }],
+    ...over,
+  };
+}
+
+function doc(fields: GroundTruthField[], lineItemCount = 1): EvalDocument {
+  return {
+    id: 'cord/test/0',
+    dataset: 'cord',
+    schema: 'receipt',
+    bytes: new Uint8Array([1]),
+    mediaType: 'image/png',
+    pages: 1,
+    fields,
+    lineItemCount,
+  };
+}
+
+describe('iou', () => {
+  it('is 1 for identical boxes and 0 for disjoint boxes', () => {
+    const a = { x0: 0, y0: 0, x1: 0.5, y1: 0.5 };
+    expect(iou(a, a)).toBe(1);
+    expect(iou(a, { x0: 0.6, y0: 0.6, x1: 1, y1: 1 })).toBe(0);
+  });
+
+  it('computes intersection over union', () => {
+    const a = { x0: 0, y0: 0, x1: 0.2, y1: 0.1 };
+    const b = { x0: 0.1, y0: 0, x1: 0.3, y1: 0.1 };
+    expect(iou(a, b)).toBeCloseTo(1 / 3);
+  });
+});
+
+describe('scoreExtraction', () => {
+  it('scores values under the field normalizer and IoU against regions', () => {
+    const document = doc([
+      gtField({ path: 'total', value: '24.000', compare: 'money' }),
+      gtField({ path: 'subtotal', value: null, compare: 'money' }),
+    ]);
+    const { fields } = scoreExtraction(document, {
+      fields: { total: leaf('24,000'), subtotal: leaf(null, { page: null, bbox: null }), lineItems: [] },
+    });
+    const total = fields.find((f) => f.path === 'total');
+    expect(total?.valueCorrect).toBe(true);
+    expect(total?.iou).toBe(1);
+    const subtotal = fields.find((f) => f.path === 'subtotal');
+    expect(subtotal?.valueCorrect).toBe(true);
+    expect(subtotal?.iou).toBeNull(); // no ground-truth region to score
+  });
+
+  it('accepts altValues', () => {
+    const document = doc([gtField({ path: 'total', value: '01/15/2021', altValues: ['Jan 15, 2021'] })]);
+    const { fields } = scoreExtraction(document, { fields: { total: leaf('jan 15, 2021') } });
+    expect(fields[0]?.valueCorrect).toBe(true);
+  });
+
+  it('scores grounding 0 for a correct value with missing bbox or wrong page', () => {
+    const document = doc([
+      gtField({ path: 'a', value: 'x' }),
+      gtField({ path: 'b', value: 'y' }),
+    ]);
+    const { fields } = scoreExtraction(document, {
+      fields: { a: leaf('x', { bbox: null }), b: leaf('y', { page: 3 }) },
+    });
+    expect(fields.find((f) => f.path === 'a')?.iou).toBe(0);
+    expect(fields.find((f) => f.path === 'b')?.iou).toBe(0);
+  });
+
+  it('leaves iou null when the value is wrong', () => {
+    const document = doc([gtField({ path: 'a', value: 'x' })]);
+    const { fields } = scoreExtraction(document, { fields: { a: leaf('wrong') } });
+    expect(fields[0]?.valueCorrect).toBe(false);
+    expect(fields[0]?.iou).toBeNull();
+  });
+
+  it('treats a missing line item as null predictions and counts extras', () => {
+    const document = doc(
+      [
+        gtField({ path: 'lineItems.0.description', value: 'present' }),
+        gtField({ path: 'lineItems.1.description', value: 'missing' }),
+      ],
+      2,
+    );
+    const { fields, extraLineItems } = scoreExtraction(document, {
+      fields: { lineItems: [{ description: leaf('present') }, undefined, { description: leaf('extra 1') }].filter(
+        (x) => x !== undefined,
+      ) },
+    });
+    expect(fields.find((f) => f.path === 'lineItems.0.description')?.valueCorrect).toBe(true);
+    expect(fields.find((f) => f.path === 'lineItems.1.description')?.valueCorrect).toBe(false);
+    expect(extraLineItems).toBe(0); // 2 predicted, 2 in ground truth
+
+    const overshoot = scoreExtraction(document, {
+      fields: {
+        lineItems: [
+          { description: leaf('present') },
+          { description: leaf('missing') },
+          { description: leaf('hallucinated') },
+        ],
+      },
+    });
+    expect(overshoot.extraLineItems).toBe(1);
+  });
+});
+
+describe('scoreFailure', () => {
+  it('marks every field incorrect, grounding 0 where scoreable', () => {
+    const document = doc([
+      gtField({ path: 'a', value: 'x' }),
+      gtField({ path: 'b', value: null }),
+    ]);
+    const { fields } = scoreFailure(document);
+    expect(fields.every((f) => !f.valueCorrect)).toBe(true);
+    expect(fields.find((f) => f.path === 'a')?.iou).toBe(0);
+    expect(fields.find((f) => f.path === 'b')?.iou).toBeNull();
+  });
+});
+
+describe('summarizeRun', () => {
+  it('aggregates accuracy, grounding, and cost per schema', () => {
+    const run: ModelRun = {
+      model: 'test-model',
+      docs: [
+        {
+          docId: 'cord/test/0',
+          dataset: 'cord',
+          schema: 'receipt',
+          error: null,
+          extraLineItems: 1,
+          usage: { inputTokens: 1000, outputTokens: 500, modelCalls: 1, costUSD: 0.01 },
+          fields: [
+            { path: 'total', compare: 'money', expected: '1', predicted: '1', valueCorrect: true, iou: 0.8, predictedPage: 0 },
+            { path: 'lineItems.0.amount', compare: 'money', expected: '2', predicted: '3', valueCorrect: false, iou: null, predictedPage: 0 },
+            { path: 'lineItems.1.amount', compare: 'money', expected: '4', predicted: '4', valueCorrect: true, iou: 0.4, predictedPage: 0 },
+          ],
+        },
+        {
+          docId: 'cord/test/1',
+          dataset: 'cord',
+          schema: 'receipt',
+          error: 'ExtractionFailedError: boom',
+          extraLineItems: 0,
+          usage: { inputTokens: 200, outputTokens: 0, modelCalls: 2, costUSD: 0.002 },
+          fields: [
+            { path: 'total', compare: 'money', expected: '1', predicted: null, valueCorrect: false, iou: 0, predictedPage: null },
+          ],
+        },
+      ],
+    };
+    const summary = summarizeRun(run);
+    const receipt = summary.perSchema.receipt;
+    expect(receipt).toBeDefined();
+    expect(receipt?.docs).toBe(2);
+    expect(receipt?.failedDocs).toBe(1);
+    expect(receipt?.fieldAccuracy).toBeCloseTo(2 / 4);
+    expect(receipt?.grounding.scoreable).toBe(3);
+    expect(receipt?.grounding.meanIoU).toBeCloseTo((0.8 + 0.4 + 0) / 3);
+    expect(receipt?.grounding.hitAt50).toBeCloseTo(1 / 3);
+    expect(receipt?.extraLineItems).toBe(1);
+    expect(receipt?.cost.totalUSD).toBeCloseTo(0.012);
+    expect(receipt?.cost.per1kDocsUSD).toBeCloseTo(6);
+    // Line-item indices collapse into one field key.
+    const amount = receipt?.perField.find((f) => f.field === 'lineItems[].amount');
+    expect(amount).toEqual({ field: 'lineItems[].amount', accuracy: 0.5, count: 2 });
+    expect(summary.perSchema.invoice).toBeUndefined();
+  });
+});
diff --git a/packages/evals/test/models.test.ts b/packages/evals/test/models.test.ts
new file mode 100644
index 0000000..903537f
--- /dev/null
+++ b/packages/evals/test/models.test.ts
@@ -0,0 +1,60 @@
+import { describe, expect, it } from 'vitest';
+import { benchmarkModels, PROVIDERS, selectProviders } from '../src/models.js';
+
+const ANTHROPIC = { ANTHROPIC_API_KEY: 'test' };
+const OPENAI = { OPENAI_API_KEY: 'test' };
+const GOOGLE = { GOOGLE_GENERATIVE_AI_API_KEY: 'test' };
+
+describe('selectProviders', () => {
+  it('runs every provider whose API key is present when EVAL_PROVIDERS is unset', () => {
+    expect(selectProviders({ ...ANTHROPIC })).toEqual(['anthropic']);
+    expect(selectProviders({ ...ANTHROPIC, ...OPENAI, ...GOOGLE })).toEqual(['anthropic', 'openai', 'google']);
+  });
+
+  it('honors an explicit EVAL_PROVIDERS selection', () => {
+    expect(selectProviders({ EVAL_PROVIDERS: 'openai', ...OPENAI })).toEqual(['openai']);
+    expect(selectProviders({ EVAL_PROVIDERS: 'google, openai', ...OPENAI, ...GOOGLE })).toEqual(['google', 'openai']);
+  });
+
+  it('deduplicates repeated providers in EVAL_PROVIDERS', () => {
+    expect(selectProviders({ EVAL_PROVIDERS: 'openai,openai', ...OPENAI })).toEqual(['openai']);
+  });
+
+  it('throws when a selected provider is missing its API key', () => {
+    expect(() => selectProviders({ EVAL_PROVIDERS: 'openai', ...ANTHROPIC })).toThrow(/OPENAI_API_KEY/);
+  });
+
+  it('throws on an unknown provider name', () => {
+    expect(() => selectProviders({ EVAL_PROVIDERS: 'grok', ...OPENAI })).toThrow(/Unknown provider "grok"/);
+  });
+
+  it('throws when no provider key is set at all', () => {
+    expect(() => selectProviders({})).toThrow(/No provider API key found/);
+  });
+});
+
+describe('benchmarkModels', () => {
+  it('defaults to the three Anthropic models when only ANTHROPIC_API_KEY is set', () => {
+    const models = benchmarkModels({ ...ANTHROPIC });
+    expect(models.map((m) => m.name)).toEqual(['claude-opus-4-8', 'claude-sonnet-5', 'claude-haiku-4-5']);
+  });
+
+  it('builds a model with pricing for every catalog entry across selected providers', () => {
+    const models = benchmarkModels({ ...ANTHROPIC, ...OPENAI, ...GOOGLE });
+    expect(models).toHaveLength(9);
+    for (const m of models) {
+      expect(m.model).toBeDefined();
+      expect(m.pricing?.inputPerMTokUSD).toBeGreaterThan(0);
+      expect(m.pricing?.outputPerMTokUSD).toBeGreaterThan(0);
+    }
+  });
+
+  it('runs only the selected provider when EVAL_PROVIDERS narrows it', () => {
+    const models = benchmarkModels({ EVAL_PROVIDERS: 'google', ...ANTHROPIC, ...GOOGLE });
+    expect(models.map((m) => m.name)).toEqual(['gemini-2.5-pro', 'gemini-2.5-flash', 'gemini-2.5-flash-lite']);
+  });
+
+  it('exposes all three providers', () => {
+    expect(PROVIDERS).toEqual(['anthropic', 'openai', 'google']);
+  });
+});
diff --git a/packages/evals/test/normalize.test.ts b/packages/evals/test/normalize.test.ts
new file mode 100644
index 0000000..81a667d
--- /dev/null
+++ b/packages/evals/test/normalize.test.ts
@@ -0,0 +1,62 @@
+import { describe, expect, it } from 'vitest';
+import { normalizeCount, normalizeMoney, normalizeText, valuesMatch } from '../src/normalize.js';
+
+describe('normalizeText', () => {
+  it('case-folds and collapses whitespace', () => {
+    expect(normalizeText('  ICE  BLACKCOFFE ')).toBe('ice blackcoffe');
+  });
+
+  it('applies NFKC normalization', () => {
+    expect(normalizeText('ＴＯＴＡＬ')).toBe('total');
+  });
+});
+
+describe('normalizeMoney', () => {
+  it('treats dots and commas as separators (Indonesian receipts)', () => {
+    expect(normalizeMoney('60.000')).toBe('60000');
+    expect(normalizeMoney('24,000')).toBe('24000');
+    expect(normalizeMoney('24000')).toBe('24000');
+  });
+
+  it('keeps the sign of negative amounts', () => {
+    expect(normalizeMoney('-60.000')).toBe('-60000');
+  });
+
+  it('strips currency symbols and prefixes', () => {
+    expect(normalizeMoney('@11000')).toBe('11000');
+    expect(normalizeMoney('Rp 91.000')).toBe('91000');
+    expect(normalizeMoney('$1,234')).toBe('1234');
+  });
+
+  it('drops leading zeros', () => {
+    expect(normalizeMoney('007')).toBe('7');
+  });
+});
+
+describe('normalizeCount', () => {
+  it('parses quantity formats seen in CORD', () => {
+    expect(normalizeCount('2')).toBe('2');
+    expect(normalizeCount('2.00')).toBe('2');
+    expect(normalizeCount('1X')).toBe('1');
+    expect(normalizeCount('1x')).toBe('1');
+  });
+
+  it('keeps fractional quantities', () => {
+    expect(normalizeCount('1.5')).toBe('1.5');
+  });
+});
+
+describe('valuesMatch', () => {
+  it('null only matches null', () => {
+    expect(valuesMatch(null, null, 'money')).toBe(true);
+    expect(valuesMatch(null, '5', 'money')).toBe(false);
+    expect(valuesMatch('5', null, 'money')).toBe(false);
+  });
+
+  it('compares under the field normalizer', () => {
+    expect(valuesMatch('24.000', '24,000', 'money')).toBe(true);
+    expect(valuesMatch('2.00', '2', 'count')).toBe(true);
+    expect(valuesMatch('JASMINE  MT ( L )', 'jasmine mt ( l )', 'text')).toBe(true);
+    expect(valuesMatch('24.000', '2400', 'money')).toBe(false);
+  });
+});
diff --git a/packages/evals/test/report.test.ts b/packages/evals/test/report.test.ts
new file mode 100644
index 0000000..492f051
--- /dev/null
+++ b/packages/evals/test/report.test.ts
@@ -0,0 +1,47 @@
+import { describe, expect, it } from 'vitest';
+import { renderBenchmarkPage, renderReadmeTable } from '../src/report.js';
+import type { RunRecord } from '../src/types.js';
+
+const record: RunRecord = {
+  startedAt: '2026-07-09T10:00:00.000Z',
+  manifestChecksum: 'abc123def456abc123def456',
+  runs: [
+    {
+      model: 'claude-sonnet-5',
+      docs: [
+        {
+          docId: 'cord/test/0',
+          dataset: 'cord',
+          schema: 'receipt',
+          error: null,
+          extraLineItems: 0,
+          usage: { inputTokens: 1000, outputTokens: 100, modelCalls: 1, costUSD: 0.0045 },
+          fields: [
+            { path: 'total', compare: 'money', expected: '1', predicted: '1', valueCorrect: true, iou: 0.9, predictedPage: 0 },
+            { path: 'tax', compare: 'money', expected: '2', predicted: null, valueCorrect: false, iou: null, predictedPage: null },
+          ],
+        },
+      ],
+    },
+  ],
+};
+
+describe('renderReadmeTable', () => {
+  it('renders one section per schema with real numbers only', () => {
+    const table = renderReadmeTable(record);
+    expect(table).toContain('Receipts — CORD-v2');
+    expect(table).not.toContain('DocILE'); // no invoice run in the record
+    expect(table).toContain('| claude-sonnet-5 | 1 | 50.0% | 100.0% | 90.0% | $4.50 |');
+  });
+});
+
+describe('renderBenchmarkPage', () => {
+  it('includes metric definitions, per-field table, and caveats', () => {
+    const page = renderBenchmarkPage(record);
+    expect(page).toContain('# extractkit benchmark');
+    expect(page).toContain('Run started 2026-07-09T10:00:00.000Z');
+    expect(page).toContain('`total`');
+    expect(page).toContain('## Caveats');
+    expect(page).toContain('CC BY 4.0');
+  });
+});
diff --git a/packages/evals/test/runner.test.ts b/packages/evals/test/runner.test.ts
new file mode 100644
index 0000000..aa6e762
--- /dev/null
+++ b/packages/evals/test/runner.test.ts
@@ -0,0 +1,116 @@
+import { MockLanguageModelV4 } from 'ai/test';
+import { describe, expect, it } from 'vitest';
+import { runModel } from '../src/runner.js';
+import type { EvalDocument, GroundTruthField } from '../src/types.js';
+
+const PNG = new Uint8Array([0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a, 0x00, 0x00, 0x00, 0x0d]);
+
+function wireLeaf(value: unknown) {
+  return value === null
+    ? { value: null, page: null, bbox: null, confidence: 0 }
+    : { value, page: 0, bbox: [0.1, 0.1, 0.5, 0.2], confidence: 0.9 };
+}
+
+function receiptEnvelope(total: string): string {
+  return JSON.stringify({
+    readable: true,
+    issues: [],
+    fields: {
+      lineItems: [
+        {
+          description: wireLeaf('TICKET CP'),
+          quantity: wireLeaf('2'),
+          unitPrice: wireLeaf(null),
+          amount: wireLeaf('60.000'),
+        },
+      ],
+      subtotal: wireLeaf(null),
+      discount: wireLeaf(null),
+      serviceCharge: wireLeaf(null),
+      tax: wireLeaf(null),
+      total: wireLeaf(total),
+    },
+  });
+}
+
+function textModel(texts: string[]) {
+  let call = 0;
+  return new MockLanguageModelV4({
+    doGenerate: async () => ({
+      content: [{ type: 'text' as const, text: texts[Math.min(call++, texts.length - 1)] as string }],
+      finishReason: { unified: 'stop' as const, raw: undefined },
+      usage: {
+        inputTokens: { total: 1000, noCache: 1000, cacheRead: undefined, cacheWrite: undefined },
+        outputTokens: { total: 100, text: 100, reasoning: undefined },
+      },
+      warnings: [],
+    }),
+  });
+}
+
+const REGION = { page: 0, bbox: { x0: 0.1, y0: 0.1, x1: 0.5, y1: 0.2 } };
+
+function receiptDoc(id: string): EvalDocument {
+  const fields: GroundTruthField[] = [
+    { path: 'lineItems.0.description', value: 'TICKET CP', compare: 'text', regions: [REGION] },
+    { path: 'lineItems.0.quantity', value: '2', compare: 'count', regions: [REGION] },
+    { path: 'lineItems.0.unitPrice', value: null, compare: 'money', regions: [] },
+    { path: 'lineItems.0.amount', value: '60.000', compare: 'money', regions: [REGION] },
+    { path: 'subtotal', value: null, compare: 'money', regions: [] },
+    { path: 'discount', value: null, compare: 'money', regions: [] },
+    { path: 'serviceCharge', value: null, compare: 'money', regions: [] },
+    { path: 'tax', value: null, compare: 'money', regions: [] },
+    { path: 'total', value: '60.000', compare: 'money', regions: [REGION] },
+  ];
+  return {
+    id,
+    dataset: 'cord',
+    schema: 'receipt',
+    bytes: PNG,
+    mediaType: 'image/png',
+    pages: 1,
+    fields,
+    lineItemCount: 1,
+  };
+}
+
+describe('runModel', () => {
+  it('extracts, scores, and reports usage with pricing', async () => {
+    const run = await runModel(
+      {
+        name: 'mock',
+        model: textModel([receiptEnvelope('60.000')]),
+        pricing: { inputPerMTokUSD: 3, outputPerMTokUSD: 15 },
+      },
+      [receiptDoc('cord/test/0')],
+    );
+    expect(run.model).toBe('mock');
+    const [doc] = run.docs;
+    expect(doc?.error).toBeNull();
+    expect(doc?.fields.every((f) => f.valueCorrect)).toBe(true);
+    expect(doc?.fields.find((f) => f.path === 'total')?.iou).toBe(1);
+    expect(doc?.extraLineItems).toBe(0);
+    expect(doc?.usage).toMatchObject({ inputTokens: 1000, outputTokens: 100, modelCalls: 1 });
+    expect(doc?.usage?.costUSD).toBeCloseTo(1000 * (3 / 1e6) + 100 * (15 / 1e6), 10);
+  });
+
+  it('records the error and scores all fields wrong when extraction fails', async () => {
+    const run = await runModel(
+      { name: 'mock', model: textModel(['not json', 'still not json']) },
+      [receiptDoc('cord/test/1')],
+    );
+    const [doc] = run.docs;
+    expect(doc?.error).toContain('ExtractionFailedError');
+    expect(doc?.fields.every((f) => !f.valueCorrect)).toBe(true);
+    // Usage from the failed attempts is still recorded.
+    expect(doc?.usage?.modelCalls).toBe(2);
+  });
+
+  it('keeps document order with concurrency', async () => {
+    const docs = ['a', 'b', 'c', 'd'].map((s) => receiptDoc(`cord/test/${s}`));
+    const run = await runModel({ name: 'mock', model: textModel([receiptEnvelope('60.000')]) }, docs, {
+      concurrency: 3,
+    });
+    expect(run.docs.map((d) => d.docId)).toEqual(docs.map((d) => d.id));
+  });
+});
diff --git a/packages/evals/tsconfig.json b/packages/evals/tsconfig.json
new file mode 100644
index 0000000..e1d180c
--- /dev/null
+++ b/packages/evals/tsconfig.json
@@ -0,0 +1,7 @@
+{
+  "extends": "../../tsconfig.base.json",
+  "compilerOptions": {
+    "types": ["node"]
+  },
+  "include": ["src", "test", "scripts", "vitest.config.ts"]
+}
diff --git a/packages/evals/vitest.config.ts b/packages/evals/vitest.config.ts
new file mode 100644
index 0000000..43e56f4
--- /dev/null
+++ b/packages/evals/vitest.config.ts
@@ -0,0 +1,7 @@
+import { defineConfig } from 'vitest/config';
+
+export default defineConfig({
+  test: {
+    include: ['test/**/*.test.ts'],
+  },
+});
diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml
index 639bed4..f10c6be 100644
--- a/pnpm-lock.yaml
+++ b/pnpm-lock.yaml
@@ -22,35 +22,106 @@ importers:
         version: 7.0.16(zod@4.4.3)
       tsdown:
         specifier: ^0.22.3
-        version: 0.22.3(typescript@6.0.3)
+        version: 0.22.3(tsx@4.23.0)(typescript@6.0.3)
       typescript:
         specifier: ^6.0.3
         version: 6.0.3
       vitest:
         specifier: ^4.1.10
-        version: 4.1.10(@types/node@26.1.0)(vite@8.1.3(@types/node@26.1.0))
+        version: 4.1.10(@types/node@26.1.0)(vite@8.1.3(@types/node@26.1.0)(esbuild@0.28.1)(tsx@4.23.0))
       zod:
         specifier: ^4.4.3
         version: 4.4.3
 
+  packages/evals:
+    dependencies:
+      '@ai-sdk/anthropic':
+        specifier: ^4.0.10
+        version: 4.0.10(zod@4.4.3)
+      '@ai-sdk/google':
+        specifier: ^4.0.12
+        version: 4.0.12(zod@4.4.3)
+      '@ai-sdk/openai':
+        specifier: ^4.0.11
+        version: 4.0.11(zod@4.4.3)
+      ai:
+        specifier: ^7.0.16
+        version: 7.0.16(zod@4.4.3)
+      extractkit:
+        specifier: workspace:*
+        version: link:../core
+      hyparquet:
+        specifier: ^1.14.0
+        version: 1.26.2
+      zod:
+        specifier: ^4.4.3
+        version: 4.4.3
+    devDependencies:
+      '@types/node':
+        specifier: ^26.1.0
+        version: 26.1.0
+      tsx:
+        specifier: ^4.19.2
+        version: 4.23.0
+      typescript:
+        specifier: ^6.0.3
+        version: 6.0.3
+      vitest:
+        specifier: ^4.1.10
+        version: 4.1.10(@types/node@26.1.0)(vite@8.1.3(@types/node@26.1.0)(esbuild@0.28.1)(tsx@4.23.0))
+
 packages:
 
+  '@ai-sdk/anthropic@4.0.10':
+    resolution: {integrity: sha512-L9GlJyL8stPyv5QTRONnVoRVw82b6iBuX4LTiRJMM++I2RfDe+yrBsPa6gz6Msll9GZ7JsX0MRNUP4G1afjDTw==}
+    engines: {node: '>=22'}
+    peerDependencies:
+      zod: ^3.25.76 || ^4.1.8
+
   '@ai-sdk/gateway@4.0.12':
     resolution: {integrity: sha512-Y7Fy8xJwPz7ZC0DhSQG3HIVk+drup42hrIj6yqKlib3CxwiR0F7nYyUI8+kPrEtbZEoyKoRstvT4/o0HEyFBHA==}
     engines: {node: '>=22'}
     peerDependencies:
       zod: ^3.25.76 || ^4.1.8
 
+  '@ai-sdk/google@4.0.12':
+    resolution: {integrity: sha512-4Rw4viFwH2Wprx6OssZzhV/KgNWaA7qYr5Kg28AlZ7r3sEvyXH0kDZtZok7onPjr0RuFTg2EjCjTIeoCesg0Ww==}
+    engines: {node: '>=22'}
+    peerDependencies:
+      zod: ^3.25.76 || ^4.1.8
+
+  '@ai-sdk/openai@4.0.11':
+    resolution: {integrity: sha512-2PLnVPBsdJusgquPVYgPWd3ox4lbUFkp7DVUSmM0lQ4VISQoNxdgo6TvFAHgwQn9wJ/ckVYVRLU7+BvHc7T8ww==}
+    engines: {node: '>=22'}
+    peerDependencies:
+      zod: ^3.25.76 || ^4.1.8
+
   '@ai-sdk/provider-utils@5.0.5':
     resolution: {integrity: sha512-oI0t3dvCoqWNV1I8o1Rybi2DXDvHES5r/TrwtJW90tuFLVepgJlftPxrcjh8vaSvjqC2diTuA2vXyjKAyHJm4A==}
     engines: {node: '>=22'}
     peerDependencies:
       zod: ^3.25.76 || ^4.1.8
 
+  '@ai-sdk/provider-utils@5.0.6':
+    resolution: {integrity: sha512-i4mVayGtC+HrRmtfPFOxvKKQgFU+1dwTTsh4MY0jY1MaGuppOwDFrNYrJ3dDXWMkO3o6FDVAKY8ZDsy7hLsO9Q==}
+    engines: {node: '>=22'}
+    peerDependencies:
+      zod: ^3.25.76 || ^4.1.8
+
+  '@ai-sdk/provider-utils@5.0.7':
+    resolution: {integrity: sha512-OSm5/5kdrHa11WIOo5LYgDKnxYWp5aB/wx5EXRHi0jpUGduMDeB6oht9U6p+UNNWIP3F/EqPpV8d7vdP/iRnqg==}
+    engines: {node: '>=22'}
+    peerDependencies:
+      zod: ^3.25.76 || ^4.1.8
+
   '@ai-sdk/provider@4.0.2':
     resolution: {integrity: sha512-pfPoy9J1B1xV7cqJ8MYHOsDYrMv5tR3+EMNfI249OhkD2uRakvav3Fo7XpD2luuN/YNCBY7KfEQc7vEV7KEtyw==}
     engines: {node: '>=22'}
 
+  '@ai-sdk/provider@4.0.3':
+    resolution: {integrity: sha512-e0CpNWJUY7OxAFAnCZkw+ri9QOHWwTs1tXP42782KFGCU07qt8NiXCrCVowyCB5dP2r5/Uls+g2oPd8kOJn9dw==}
+    engines: {node: '>=22'}
+
   '@babel/generator@8.0.0':
     resolution: {integrity: sha512-NT9NrVwJsbSV6Y2FSstWa71EETOnzrjkL5/wX3D2mYHtKM+qvqB1DvR4D0Setb/gDBsHzRICifwEWMO8CnTF6g==}
     engines: {node: ^22.18.0 || >=24.11.0}
@@ -81,6 +152,162 @@ packages:
   '@emnapi/wasi-threads@1.2.2':
     resolution: {integrity: sha512-c95qOXkHdydNKhscBTebqEC1CVAZpyqOfVfBzQ1qgzyl3gfeldUjIggDbIZgDKsHLgnsM+igH7TJ/eAasaVuMA==}
 
+  '@esbuild/aix-ppc64@0.28.1':
+    resolution: {integrity: sha512-Svl7tq8k/08+p6CXPpRjQ1fKX+1odH/BQbb48fV6fj3CWHhsoIOoY87w1oHXm0qEpkIK3ZfVgp0hed3XBXzXMQ==}
+    engines: {node: '>=18'}
+    cpu: [ppc64]
+    os: [aix]
+
+  '@esbuild/android-arm64@0.28.1':
+    resolution: {integrity: sha512-34EGEbCIAgosYz6goLcopX6Mo7NyGv9tfwEM2/7Ce2VcVRk568iSvniGWcUXIy7wEDR1wzolcxcriFVrWYcwBg==}
+    engines: {node: '>=18'}
+    cpu: [arm64]
+    os: [android]
+
+  '@esbuild/android-arm@0.28.1':
+    resolution: {integrity: sha512-0k2F129Xdio1TdJfzJ8sy1Q47vUD2NnwdhiAf7drUN1EBTfPf4hsFCtmMgu/6m8JSzsBrlmVjudMBQqOfG8usQ==}
+    engines: {node: '>=18'}
+    cpu: [arm]
+    os: [android]
+
+  '@esbuild/android-x64@0.28.1':
+    resolution: {integrity: sha512-dbwY7ltSMDWsRatcRpCnES4F+im88OCUgGZjy52shC7GqHRE/cYlxNbB4Z4UpJswpcc4Qxd2oE/ufM0p61IKng==}
+    engines: {node: '>=18'}
+    cpu: [x64]
+    os: [android]
+
+  '@esbuild/darwin-arm64@0.28.1':
+    resolution: {integrity: sha512-TZbWkQY7kvTAXbXUT7uVACR5cMHsDiSz9z7ZKAX/RTq/WJEk3QyRr0wZpNhBDX+/0CtdqUIJlOiodQcta6tY3Q==}
+    engines: {node: '>=18'}
+    cpu: [arm64]
+    os: [darwin]
+
+  '@esbuild/darwin-x64@0.28.1':
+    resolution: {integrity: sha512-zfdzgK9ACBNZLI/CyHTOx81SyNbM6YXn7rxSgX97VjyiPl9W1i4Ka4fgKECEoFCKGpvBj5qArWIGgQjOwkgskQ==}
+    engines: {node: '>=18'}
+    cpu: [x64]
+    os: [darwin]
+
+  '@esbuild/freebsd-arm64@0.28.1':
+    resolution: {integrity: sha512-wG2EA8ENdEI0qhkSZMjfqrdY+ziCYCPMmtZjjIwOmXFjmyzEHn+UUxk5of+SYsjtfs3VpnlC7QLzSI5hY/rOAw==}
+    engines: {node: '>=18'}
+    cpu: [arm64]
+    os: [freebsd]
+
+  '@esbuild/freebsd-x64@0.28.1':
+    resolution: {integrity: sha512-i7dZ9vQgnvSCzi/rYCXNgtF/U+eKZNJBzu3eTQbRgHnM7tNSizLOkRFAl3qzVc/Op/u5YkHHa4pf/3DOYHthLQ==}
+    engines: {node: '>=18'}
+    cpu: [x64]
+    os: [freebsd]
+
+  '@esbuild/linux-arm64@0.28.1':
+    resolution: {integrity: sha512-yHs+0uc8+nvEAfAfxrWQKK5peSNzBc4PegcMO0EJ2hT71uA7vB8Ihg2e77R2P7SG5uYjPbHlLLmve4LLLRCf0g==}
+    engines: {node: '>=18'}
+    cpu: [arm64]
+    os: [linux]
+
+  '@esbuild/linux-arm@0.28.1':
+    resolution: {integrity: sha512-qVXBOHQS+d5Y722GwJzJUtOLlX7km3CraOaGormF1pDtPd2C/l1SHRPgjLunLGe51Sh5YYWKMFDyV4SxgMQYTQ==}
+    engines: {node: '>=18'}
+    cpu: [arm]
+    os: [linux]
+
+  '@esbuild/linux-ia32@0.28.1':
+    resolution: {integrity: sha512-d1z4ZuP0ajrfz/FhGT4vv278rX8KnPPJx8i5+AtK7TYbx9Le9F1hyzurZpkEyjkGa9dUGhQow4C1NmeGvqxN2w==}
+    engines: {node: '>=18'}
+    cpu: [ia32]
+    os: [linux]
+
+  '@esbuild/linux-loong64@0.28.1':
+    resolution: {integrity: sha512-M5sRjUVZrkm1OAPR3dlOYzNmN+loZKGVi1VUQGrwuqLcbR6qeAz+famMhjASeH3YVKvZz+zT1jlh/keC3Rj/lg==}
+    engines: {node: '>=18'}
+    cpu: [loong64]
+    os: [linux]
+
+  '@esbuild/linux-mips64el@0.28.1':
+    resolution: {integrity: sha512-mRObBZeHh2OxcBFPWE/FjylkRgZdYuiTR3vaTozquCGOH14iP9oN4x4Ge81CoIDYQrXmIxpFumJBu5MtZpnQJQ==}
+    engines: {node: '>=18'}
+    cpu: [mips64el]
+    os: [linux]
+
+  '@esbuild/linux-ppc64@0.28.1':
+    resolution: {integrity: sha512-slScBsMAb3GFDcdrCgLwZtPYRoH2H/youv10QiZyRjmsP48fznoveWytSgCI/R0ZcUgpc0ZhIUEx6LHts8yrfQ==}
+    engines: {node: '>=18'}
+    cpu: [ppc64]
+    os: [linux]
+
+  '@esbuild/linux-riscv64@0.28.1':
+    resolution: {integrity: sha512-kw0owk1o0GFETUJyW0jc0G4Yzs0BHZn0JDZ8JRT088vjJYX777BAs1fDGxAC+q831qOs2DTC96mNsG2opdfyyQ==}
+    engines: {node: '>=18'}
+    cpu: [riscv64]
+    os: [linux]
+
+  '@esbuild/linux-s390x@0.28.1':
+    resolution: {integrity: sha512-/lAIjX8aYFRByhh6L5rYtPEDRqa9de/4V/juOXcta5frjvzXO4/sqEtyytse0g3zZFuWu5cDN0MkLz2qRDD2Ag==}
+    engines: {node: '>=18'}
+    cpu: [s390x]
+    os: [linux]
+
+  '@esbuild/linux-x64@0.28.1':
+    resolution: {integrity: sha512-u/anNYF2mmVOEDwLtnQ1wOr3EZ9sTNGLWrsYGYwHWzGA3Si84IOkHXlbWTD1NB+9/1lcnweYKO54uhxZydNzfA==}
+    engines: {node: '>=18'}
+    cpu: [x64]
+    os: [linux]
+
+  '@esbuild/netbsd-arm64@0.28.1':
+    resolution: {integrity: sha512-oks0DYbLwWMmaakTsCb+zL4E+aHRVLom9IJZOAthMQEPiQmydXHkziYEsGYRx0uNV/IjEKGAV941JzH02pflqw==}
+    engines: {node: '>=18'}
+    cpu: [arm64]
+    os: [netbsd]
+
+  '@esbuild/netbsd-x64@0.28.1':
+    resolution: {integrity: sha512-aeL6lAnN89Hz43Mlh1G8ARasbuoYvSITDEx0tHh5b7jJnHcssqgjy9Yx430GDpmCa6OyrKoS0aNRjKundRizGg==}
+    engines: {node: '>=18'}
+    cpu: [x64]
+    os: [netbsd]
+
+  '@esbuild/openbsd-arm64@0.28.1':
+    resolution: {integrity: sha512-MEFJe5C3R8pwXdZ5Y21oo6m7ePiS0d9pWucn99O/wvyJZChoIQKrQDxKrGeW8F5+T0okTHesAmDeiHDTIq0V/Q==}
+    engines: {node: '>=18'}
+    cpu: [arm64]
+    os: [openbsd]
+
+  '@esbuild/openbsd-x64@0.28.1':
+    resolution: {integrity: sha512-i/ZLIOafE0Z8cI/XANJAixoJL/uRAoS2xOA3rb0xN+KK0K177cMAsQYkzHtBrtMXAKuAc7HGgcWiZ/sRC1Nxgw==}
+    engines: {node: '>=18'}
+    cpu: [x64]
+    os: [openbsd]
+
+  '@esbuild/openharmony-arm64@0.28.1':
+    resolution: {integrity: sha512-ge+Z7EXFNt2BO1oAMsVpiQ8EwndV9i1xXerAeTIK7AtPs3bKFXQM7nlRxDSIUIMeueR1CNXxqztLzdNeReKBJg==}
+    engines: {node: '>=18'}
+    cpu: [arm64]
+    os: [openharmony]
+
+  '@esbuild/sunos-x64@0.28.1':
+    resolution: {integrity: sha512-BEjgtECkL3vY+SaSQ6nzVfiALUeFxpawyp8Jmf5PtYhf1Ug40N1h/hxlhts+f1FvSvarEigdxS3BlSMI2PJLcQ==}
+    engines: {node: '>=18'}
+    cpu: [x64]
+    os: [sunos]
+
+  '@esbuild/win32-arm64@0.28.1':
+    resolution: {integrity: sha512-lCv9eK/H6ZJWbE7bh2nw54CZ9M2nupBxJcTsdk/QQnWkdSjKGuxmmH8/GWrlT1eMmZfn4dGcCjRte397WqfQXA==}
+    engines: {node: '>=18'}
+    cpu: [arm64]
+    os: [win32]
+
+  '@esbuild/win32-ia32@0.28.1':
+    resolution: {integrity: sha512-zvb/mB2bSCoJOpoCBgYKKpX6YM6mJBlBUVUtVj41DlZJVEB6/0CKlRYxP5wWl1C1ILiCoAU5wZZ4q1P3qeS6Eg==}
+    engines: {node: '>=18'}
+    cpu: [ia32]
+    os: [win32]
+
+  '@esbuild/win32-x64@0.28.1':
+    resolution: {integrity: sha512-bm4Mowrv+GXMlpWX++EcXw/iLyd1o3+bJkC2DkWXYVvgZCqD/bSj9ctZeAMC3cIxgjRVR2Dufaiu4YPxr5gW1A==}
+    engines: {node: '>=18'}
+    cpu: [x64]
+    os: [win32]
+
   '@jridgewell/gen-mapping@0.3.13':
     resolution: {integrity: sha512-2kkt/7niJ6MgEPxF0bYdQ6etZaA+fQvDcLKckhy1yIQOzaoKjBBjSj63/aLVjYE3qhRt5dvM+uUyfCg6UKCBbA==}
 
@@ -316,6 +543,11 @@ packages:
   es-module-lexer@2.3.0:
     resolution: {integrity: sha512-KLdwQm2NvGLDkQDCGvmiQrhkd0JbMzXthwQAUgWjQuQdBLFa3eiBP5arXZyA+f8x+x7OXgud6bq2rxjGtHV2tw==}
 
+  esbuild@0.28.1:
+    resolution: {integrity: sha512-HrJrvZv5ayxBzPfwphOoNzkzOIIlifzk0KJrGK2c8R4+LKpMtpYLQeUdjnwjWv/LZlkH2laZk+4w78pi99D4Vw==}
+    engines: {node: '>=18'}
+    hasBin: true
+
   estree-walker@3.0.3:
     resolution: {integrity: sha512-7RUKfXgSMMkzt6ZuXmqapOurLGPPfgj6l9uRZ7lRGolvk0y2yocc35LdcxKC5PQZdn2DMqioAQ2NoWcrTKmm6g==}
 
@@ -348,6 +580,9 @@ packages:
   hookable@6.1.1:
     resolution: {integrity: sha512-U9LYDy1CwhMCnprUfeAZWZGByVbhd54hwepegYTK7Pi5NvqEj63ifz5z+xukznehT7i6NIZRu89Ay1AZmRsLEQ==}
 
+  hyparquet@1.26.2:
+    resolution: {integrity: sha512-6qjyK7R2tZ4vnYP1J8NpcCeDPK1UIYk3xh5XgtQUnUM1iwkYbvI6Mz3xtmpMd6AfCCeoUoJVbEq29k5GReZRWA==}
+
   import-without-cache@0.4.0:
     resolution: {integrity: sha512-NkJQA7oZ4YHQhd2+H3BoRFKF3d/XNsiKpHZCQEMH9pDX27hQQLsTyOocyRgaIVtf8gHX3Nt3LPkR4e5EdtPAGQ==}
     engines: {node: ^22.18.0 || >=24.0.0}
@@ -569,6 +804,11 @@ packages:
   tslib@2.8.1:
     resolution: {integrity: sha512-oJFu94HQb+KVduSUQL7wnpmqnfmLsOA/nAh6b6EH0wCEoK0/mPeXU6c3wKDV83MkOuHPRHtSXKKU99IBazS/2w==}
 
+  tsx@4.23.0:
+    resolution: {integrity: sha512-eUdUIaCr963q2h5u3+QwvYp0+eqPvn+egeqZUm0hwERCqqx1E3kK5ehbGCvqSE5MQAULr67ww0cA3jKc3YkM1w==}
+    engines: {node: '>=18.0.0'}
+    hasBin: true
+
   typescript@6.0.3:
     resolution: {integrity: sha512-y2TvuxSZPDyQakkFRPZHKFm+KKVqIisdg9/CZwm9ftvKXLP8NRWj38/ODjNbr43SsoXqNuAisEf1GdCxqWcdBw==}
     engines: {node: '>=14.17'}
@@ -674,6 +914,12 @@ packages:
 
 snapshots:
 
+  '@ai-sdk/anthropic@4.0.10(zod@4.4.3)':
+    dependencies:
+      '@ai-sdk/provider': 4.0.2
+      '@ai-sdk/provider-utils': 5.0.6(zod@4.4.3)
+      zod: 4.4.3
+
   '@ai-sdk/gateway@4.0.12(zod@4.4.3)':
     dependencies:
       '@ai-sdk/provider': 4.0.2
@@ -681,6 +927,18 @@ snapshots:
       '@vercel/oidc': 3.2.0
       zod: 4.4.3
 
+  '@ai-sdk/google@4.0.12(zod@4.4.3)':
+    dependencies:
+      '@ai-sdk/provider': 4.0.3
+      '@ai-sdk/provider-utils': 5.0.7(zod@4.4.3)
+      zod: 4.4.3
+
+  '@ai-sdk/openai@4.0.11(zod@4.4.3)':
+    dependencies:
+      '@ai-sdk/provider': 4.0.3
+      '@ai-sdk/provider-utils': 5.0.7(zod@4.4.3)
+      zod: 4.4.3
+
   '@ai-sdk/provider-utils@5.0.5(zod@4.4.3)':
     dependencies:
       '@ai-sdk/provider': 4.0.2
@@ -689,10 +947,30 @@ snapshots:
       eventsource-parser: 3.1.0
       zod: 4.4.3
 
+  '@ai-sdk/provider-utils@5.0.6(zod@4.4.3)':
+    dependencies:
+      '@ai-sdk/provider': 4.0.2
+      '@standard-schema/spec': 1.1.0
+      '@workflow/serde': 4.1.0
+      eventsource-parser: 3.1.0
+      zod: 4.4.3
+
+  '@ai-sdk/provider-utils@5.0.7(zod@4.4.3)':
+    dependencies:
+      '@ai-sdk/provider': 4.0.3
+      '@standard-schema/spec': 1.1.0
+      '@workflow/serde': 4.1.0
+      eventsource-parser: 3.1.0
+      zod: 4.4.3
+
   '@ai-sdk/provider@4.0.2':
     dependencies:
       json-schema: 0.4.0
 
+  '@ai-sdk/provider@4.0.3':
+    dependencies:
+      json-schema: 0.4.0
+
   '@babel/generator@8.0.0':
     dependencies:
       '@babel/parser': 8.0.0
@@ -731,6 +1009,84 @@ snapshots:
       tslib: 2.8.1
     optional: true
 
+  '@esbuild/aix-ppc64@0.28.1':
+    optional: true
+
+  '@esbuild/android-arm64@0.28.1':
+    optional: true
+
+  '@esbuild/android-arm@0.28.1':
+    optional: true
+
+  '@esbuild/android-x64@0.28.1':
+    optional: true
+
+  '@esbuild/darwin-arm64@0.28.1':
+    optional: true
+
+  '@esbuild/darwin-x64@0.28.1':
+    optional: true
+
+  '@esbuild/freebsd-arm64@0.28.1':
+    optional: true
+
+  '@esbuild/freebsd-x64@0.28.1':
+    optional: true
+
+  '@esbuild/linux-arm64@0.28.1':
+    optional: true
+
+  '@esbuild/linux-arm@0.28.1':
+    optional: true
+
+  '@esbuild/linux-ia32@0.28.1':
+    optional: true
+
+  '@esbuild/linux-loong64@0.28.1':
+    optional: true
+
+  '@esbuild/linux-mips64el@0.28.1':
+    optional: true
+
+  '@esbuild/linux-ppc64@0.28.1':
+    optional: true
+
+  '@esbuild/linux-riscv64@0.28.1':
+    optional: true
+
+  '@esbuild/linux-s390x@0.28.1':
+    optional: true
+
+  '@esbuild/linux-x64@0.28.1':
+    optional: true
+
+  '@esbuild/netbsd-arm64@0.28.1':
+    optional: true
+
+  '@esbuild/netbsd-x64@0.28.1':
+    optional: true
+
+  '@esbuild/openbsd-arm64@0.28.1':
+    optional: true
+
+  '@esbuild/openbsd-x64@0.28.1':
+    optional: true
+
+  '@esbuild/openharmony-arm64@0.28.1':
+    optional: true
+
+  '@esbuild/sunos-x64@0.28.1':
+    optional: true
+
+  '@esbuild/win32-arm64@0.28.1':
+    optional: true
+
+  '@esbuild/win32-ia32@0.28.1':
+    optional: true
+
+  '@esbuild/win32-x64@0.28.1':
+    optional: true
+
   '@jridgewell/gen-mapping@0.3.13':
     dependencies:
       '@jridgewell/sourcemap-codec': 1.5.5
@@ -850,13 +1206,13 @@ snapshots:
       chai: 6.2.2
       tinyrainbow: 3.1.0
 
-  '@vitest/mocker@4.1.10(vite@8.1.3(@types/node@26.1.0))':
+  '@vitest/mocker@4.1.10(vite@8.1.3(@types/node@26.1.0)(esbuild@0.28.1)(tsx@4.23.0))':
     dependencies:
       '@vitest/spy': 4.1.10
       estree-walker: 3.0.3
       magic-string: 0.30.21
     optionalDependencies:
-      vite: 8.1.3(@types/node@26.1.0)
+      vite: 8.1.3(@types/node@26.1.0)(esbuild@0.28.1)(tsx@4.23.0)
 
   '@vitest/pretty-format@4.1.10':
     dependencies:
@@ -919,6 +1275,35 @@ snapshots:
 
   es-module-lexer@2.3.0: {}
 
+  esbuild@0.28.1:
+    optionalDependencies:
+      '@esbuild/aix-ppc64': 0.28.1
+      '@esbuild/android-arm': 0.28.1
+      '@esbuild/android-arm64': 0.28.1
+      '@esbuild/android-x64': 0.28.1
+      '@esbuild/darwin-arm64': 0.28.1
+      '@esbuild/darwin-x64': 0.28.1
+      '@esbuild/freebsd-arm64': 0.28.1
+      '@esbuild/freebsd-x64': 0.28.1
+      '@esbuild/linux-arm': 0.28.1
+      '@esbuild/linux-arm64': 0.28.1
+      '@esbuild/linux-ia32': 0.28.1
+      '@esbuild/linux-loong64': 0.28.1
+      '@esbuild/linux-mips64el': 0.28.1
+      '@esbuild/linux-ppc64': 0.28.1
+      '@esbuild/linux-riscv64': 0.28.1
+      '@esbuild/linux-s390x': 0.28.1
+      '@esbuild/linux-x64': 0.28.1
+      '@esbuild/netbsd-arm64': 0.28.1
+      '@esbuild/netbsd-x64': 0.28.1
+      '@esbuild/openbsd-arm64': 0.28.1
+      '@esbuild/openbsd-x64': 0.28.1
+      '@esbuild/openharmony-arm64': 0.28.1
+      '@esbuild/sunos-x64': 0.28.1
+      '@esbuild/win32-arm64': 0.28.1
+      '@esbuild/win32-ia32': 0.28.1
+      '@esbuild/win32-x64': 0.28.1
+
   estree-walker@3.0.3:
     dependencies:
       '@types/estree': 1.0.9
@@ -940,6 +1325,8 @@ snapshots:
 
   hookable@6.1.1: {}
 
+  hyparquet@1.26.2: {}
+
   import-without-cache@0.4.0: {}
 
   jsesc@3.1.0: {}
@@ -1088,7 +1475,7 @@ snapshots:
 
   tree-kill@1.2.2: {}
 
-  tsdown@0.22.3(typescript@6.0.3):
+  tsdown@0.22.3(tsx@4.23.0)(typescript@6.0.3):
     dependencies:
       ansis: 4.3.1
       cac: 7.0.0
@@ -1106,6 +1493,7 @@ snapshots:
       tree-kill: 1.2.2
       unconfig-core: 7.5.0
     optionalDependencies:
+      tsx: 4.23.0
       typescript: 6.0.3
     transitivePeerDependencies:
       - '@ts-macro/tsc'
@@ -1118,6 +1506,12 @@ snapshots:
   tslib@2.8.1:
     optional: true
 
+  tsx@4.23.0:
+    dependencies:
+      esbuild: 0.28.1
+    optionalDependencies:
+      fsevents: 2.3.3
+
   typescript@6.0.3: {}
 
   unconfig-core@7.5.0:
@@ -1127,7 +1521,7 @@ snapshots:
 
   undici-types@8.3.0: {}
 
-  vite@8.1.3(@types/node@26.1.0):
+  vite@8.1.3(@types/node@26.1.0)(esbuild@0.28.1)(tsx@4.23.0):
     dependencies:
       lightningcss: 1.32.0
       picomatch: 4.0.5
@@ -1136,12 +1530,14 @@ snapshots:
       tinyglobby: 0.2.17
     optionalDependencies:
       '@types/node': 26.1.0
+      esbuild: 0.28.1
       fsevents: 2.3.3
+      tsx: 4.23.0
 
-  vitest@4.1.10(@types/node@26.1.0)(vite@8.1.3(@types/node@26.1.0)):
+  vitest@4.1.10(@types/node@26.1.0)(vite@8.1.3(@types/node@26.1.0)(esbuild@0.28.1)(tsx@4.23.0)):
     dependencies:
       '@vitest/expect': 4.1.10
-      '@vitest/mocker': 4.1.10(vite@8.1.3(@types/node@26.1.0))
+      '@vitest/mocker': 4.1.10(vite@8.1.3(@types/node@26.1.0)(esbuild@0.28.1)(tsx@4.23.0))
       '@vitest/pretty-format': 4.1.10
       '@vitest/runner': 4.1.10
       '@vitest/snapshot': 4.1.10
@@ -1158,7 +1554,7 @@ snapshots:
       tinyexec: 1.2.4
       tinyglobby: 0.2.17
       tinyrainbow: 3.1.0
-      vite: 8.1.3(@types/node@26.1.0)
+      vite: 8.1.3(@types/node@26.1.0)(esbuild@0.28.1)(tsx@4.23.0)
       why-is-node-running: 2.3.0
     optionalDependencies:
       '@types/node': 26.1.0