Frontier coding agents write GPU kernels. Each session is one autonomous agent, graded against a roofline (or ms/speedup) ceiling, then reward-hack audited before anything is published. Live: kernelbench.com.
This monorepo is the website and the evals. GPU sessions launch to
Lambda / Brev (or another remote worker). Operator workflow: AGENTS.md.
Methodology and history: each bench's SPEC.md and DEVLOG.md.
| bench | path | what | site |
|---|---|---|---|
| hard | benchmarks/hard/ |
per-op kernels (CUDA or Triton), roofline-graded | /hard |
| mega | benchmarks/mega/ |
full fused megakernels | /mega |
| cuda | benchmarks/cuda/ |
CUDA-only writing deck (Triton/DSL fail) | /cuda |
| mini | benchmarks/mini/ |
small-model (<200B) deck, capped + 5-repeat (WIP) | unpublished |
| multi | benchmarks/multi/ |
4×H100 NVLink multi-GPU (WIP, frontier roster) | unpublished |
| v3 | benchmarks/v3/ |
offline archive (separate harness) | not on site |
Hard / mega / cuda share harness machinery and run unlimited wall-clock
(BUDGET_SECONDS=0). Mini is capped; multi is sequential on a 4-GPU node.
Next.js 16 + Tailwind. Package manager is bun (bun.lock):
bun install
bun dev # http://localhost:3000
bun run buildSite data is baked at build time from benchmarks/*/results/
(app/_lib/data.ts). Publish/deploy: kb publish then kb deploy — see
AGENTS.md.
AGENTS.md— sweeps, harnesses, Lambda/Brev, publish, auditsbenchmarks/<bench>/SPEC.md— methodologybenchmarks/<bench>/DEVLOG.md— design historybenchmarks/<bench>/README.md— short human entry for that deck