Home
Apple Silicon port of karpathy/autoresearch with autonomous LLM-driven experiment optimization across multiple training datasets.
- TUI Dashboard — Real-time terminal dashboard for monitoring training runs
- Multi-Dataset Suite — Framework for running experiments across different training datasets
Five datasets tested on the same hardware (Apple M5 Max, 64 GB) — architecture converges to AR=32 across all datasets. Three of five converge to the exact same hyperparameters. Both FineWeb-Edu variants diverge, confirming that educational text's hyperparameter divergence is data-dependent, not path-dependent.
| Date | Chip | Dataset | Experiments | Best val_bpb | Branch |
|---|---|---|---|---|---|
| Mar 21, 2026 — FineWeb-Edu-High (101 experiments) | Apple M5 Max (64 GB) | FineWeb-Edu-High | 101 | 1.346 | autoresearch/mar20-fineweb-edu-high |
| Mar 20, 2026 — SlimPajama (101 experiments) | Apple M5 Max (64 GB) | SlimPajama | 101 | 1.526 | autoresearch/mar20-slimpajama |
| Mar 20, 2026 — Cosmopedia-v2 (103 experiments) | Apple M5 Max (64 GB) | Cosmopedia-v2 | 103 | 0.961 | autoresearch/mar19-cosmopedia-v2 |
| Mar 19, 2026 — Climbmix (101 experiments) | Apple M5 Max (64 GB) | climbmix-400b | 101 | 1.296 | autoresearch/mar19-climbmix |
| Mar 17, 2026 — FineWeb-Edu (88 experiments) | Apple M5 Max (64 GB) | FineWeb-Edu 10BT | 88 | 1.342 | master |
| Mar 16, 2026 — Climbmix (81 experiments) | Apple M5 Max (64 GB) | climbmix-400b | 81 | 1.335 | autoresearch/mar16-agent |
| Date | Chip | Best val_bpb | Branch |
|---|---|---|---|
| Mar 15, 2026 — M5 Max | Apple M5 Max (64 GB) | 1.320 | autoresearch/mar14-m5max |
| Mar 14, 2026 — M4 Pro | Apple M4 Pro (24 GB) | 1.429 | autoresearch/mar14 |
| Mar 11, 2026 — M1 Max | Apple M1 Max (64 GB) | 1.621 | autoresearch/mar11 |
-
karpathy/autoresearch PR #303 — "Evaluating Experiment Results at Scale" by Dean Sharon. Guide for noise floor estimation, Pareto efficiency, and reading results.tsv at scale. Adapted for Apple Silicon in
docs/evaluating-results.md.