A $1,999 AMD laptop outscored a $1,400 GPU (in a $2,500+ system) on ML training quality. Here's the data.
An autonomous AI research agent ran 200+ experiments across 5 rounds on an AMD Radeon 8060S (Strix Halo integrated GPU, 128GB unified memory, 54W total system power). It found a recipe that achieves val_bpb 1.2153 — 34% better than the same recipe on an NVIDIA RTX 4090 (val_bpb 1.844). Two models were trained using 29.7 GB VRAM — impossible on a 4090's 24 GB hard limit.
This is not a benchmark war. This is a message:
To NVIDIA: Your ecosystem advantage is real, but your hardware isn't untouchable.
To AMD: Your silicon works. Your software stack is why people stay on NVIDIA. Fix it.
Let's be honest about what each setup actually costs:
| Component | AMD Strix Halo | NVIDIA 4090 Build |
|---|---|---|
| GPU | Integrated (included) | RTX 4090: ~$1,400 |
| CPU | Ryzen AI MAX+ 395 (included) | Comparable CPU: ~$350 |
| Memory | 128 GB unified (included) | 64 GB DDR5: ~$180 |
| Motherboard | Integrated (included) | ATX board: ~$180 |
| PSU | 54W laptop charger (included) | 850W+ PSU: ~$130 |
| Case + storage | Laptop (included) | Case + NVMe: ~$150 |
| Total | $1,999 | ~$2,390+ |
| Power at wall | 54W | 550W+ system |
| Electricity (8hr/day, 1yr) | ~$19/yr | ~$193/yr |
The Strix Halo is a complete system — screen, keyboard, battery, WiFi. The 4090 build is a desktop tower that needs a monitor, costs $400 more, and drinks 10x the power. The AMD system has 128 GB unified memory (allocatable up to 96 GB VRAM, ~112 GB with Linux kernel hacks) vs the 4090's hard 24 GB VRAM ceiling.
| Metric | AMD Radeon 8060S | NVIDIA RTX 4090 |
|---|---|---|
| val_bpb | 1.2153 | 1.844 |
| System cost | $1,999 (complete) | ~$2,390+ (tower only) |
| Power draw | 54W system | 550W+ system |
| Throughput (tok/sec) | 51,000 | 320,681 |
| MFU (utilization) | 25% | 7.7% |
| VRAM available | 96 GB (up to 112 GB) | 24 GB (hard limit) |
| VRAM used | 45 GB / 128 GB | 8.8 GB / 24 GB |
| SDPA Flash (ms) | 2.3 | 0.23 |
| Matmul 4096 (TFLOPS) | 30.6 | 155.0 |
| bf16 stability bugs | yes (documented) | none |
NVIDIA is 6x faster on throughput. AMD gets 3x better hardware utilization, 4x the memory ceiling, and a 34% better training score on the same recipe in the same 5-minute budget.
The recipe was optimized over 200+ autonomous experiments across 5 rounds specifically on AMD hardware. Key findings:
- Round 3 (33 experiments): Ablation identified 2 CRITICAL hyperparams — HEAD_DIM 64 (+1.11% regression when reverted), WARMDOWN_RATIO 0.7 (+1.08% regression). 5 "improvements" debunked as noise.
- Round 4 (40 experiments): Aspect ratio, depth, window pattern, LR sweeps. Memory wall proof — trained a model at 29.7 GB VRAM (impossible on 4090).
- Round 5 (60+ experiments): Deep squeeze on every remaining hyperparameter. 6 LR sweeps, weight decay, Adam betas, batch size, interaction effects, extended training, memory wall v2. Best val_bpb improved from 1.227 → 1.2153. Two models trained beyond 4090's 24 GB limit. beta2 < 0.97 causes NaN — another bf16 bug for AMD's list.
Hardware-specific tuning matters more than raw throughput for research quality.
15 years of ecosystem investment:
- pip install works the first time
- Flash Attention 2 auto-selects via SDPA
- Stable PyTorch releases (not nightlies)
- Pre-built kernels for everything
This isn't a hardware gap. It's a software gap.
We found real bugs. AMD engineering, please fix these:
- bf16 accumulation breaks at small batch sizes —
TOTAL_BATCH_SIZE=2^13always NaN/crashes. Gradient accumulation precision issue. - bf16 breaks at small head dimensions —
HEAD_DIM=32produces NaN. Kernel-level precision issue. - Deep networks unreliable —
DEPTH=12+timeout/crash. bf16 accumulation in deep networks. - Wide aspect ratios crash —
ASPECT_RATIO=128timeout. Possible memory layout issue. - Matrix LR cliff —
MATRIX_LR=0.15works,0.20is dead. Sharp boundary. - Adam beta2 < 0.97 produces NaN — Optimizer precision issue. beta2=0.95 and 0.96 both crash. (Round 5)
- bf16 accumulation degrades over extended training — 10-minute runs hit NaN on run 3/3 at ~1008 steps. Precision erodes over time. (Round 5)
- TheROCk nightlies required — Stable ROCm does not ship gfx1151 kernels. Consumer Strix Halo users MUST use the nightly index at
https://rocm.nightlies.amd.com/v2/gfx1151/. - AOTriton flash attention hidden behind env var —
TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1gives 19x SDPA speedup (44ms to 2.3ms). This is not documented. This is not default. This should be both. PYTORCH_HIP_ALLOC_CONF=backend:malloccrashes PyTorch — Set by default in some ROCm shell profiles. Must be manually unset. A default that crashes your own framework is a bug.- No stable PyTorch wheels for gfx1151 — Users must install from nightlies:
torch-2.11.0a0+rocm7.11.0a20260106. Ship stable wheels.
- System: GMKTEC EVO X2 mini PC — $1,999 (Strix Halo platform)
- APU: AMD Ryzen AI MAX+ 395
- GPU: Radeon 8060S (gfx1151) — integrated
- Memory: 128 GB unified (shared CPU/GPU, up to 96 GB allocatable as VRAM, ~112 GB with Linux kernel hacks)
- TDP: 54W total system
- PyTorch: 2.11.0a0+rocm7.11.0a20260106 (TheROCk nightly)
- Attention: AOTriton experimental via SDPA
- GPU: NVIDIA GeForce RTX 4090 — discrete
- VRAM: 24 GB GDDR6X
- TDP: 450W GPU
- Host CPU: AMD EPYC 7452 32-Core
- PyTorch: 2.4.1+cu124 (stable)
- Attention: FlashAttention 2 via SDPA (FA3 unavailable — needs SM90/Hopper)
An AI coding agent (Claude Code on Opus) ran the experiment loop from karpathy/autoresearch: modify train.py, train 5 minutes, keep improvements, discard regressions, repeat.
| Milestone | val_bpb | Improvement |
|---|---|---|
| Baseline (Karpathy default) | 1.819 | — |
| Batch size tuning (2^20 → 2^15) | 1.295 | -28.8% |
| SwiGLU + full attention | 1.292 | -29.0% |
| Warmdown schedule | 1.264 | -30.5% |
| LR + weight decay tuning | 1.256 | -31.0% |
| Head dim + unembed LR | 1.255 | -31.0% |
| ASPECT_RATIO 32 (Round 3) | 1.227 | -32.5% |
| Aspect/warmdown/matrix LR (Round 4) | 1.2189 | -33.0% |
| Deep squeeze — all hyperparams (Round 5) | 1.2153 | -33.2% |
- 33 experiments in 4 hours (autonomous agent, zero human intervention)
- Confirmation band: 0.007 variance across 5 identical runs — recipe is reproducible
- 2 CRITICAL hyperparams identified via ablation (HEAD_DIM 64, WARMDOWN_RATIO 0.7)
- 5 "improvements" debunked as noise (SCALAR_LR, ADAM_BETAS, FINAL_LR_FRAC, MATRIX_LR, UNEMBEDDING_LR)
- Full failure boundary map with exact reproduction steps
See round3_report.md for complete data.
- 40 experiments — aspect ratio, head dim, warmdown, matrix LR, depth, window pattern sweeps
- Best val_bpb 1.2189 (ASPECT_RATIO=40, WARMDOWN=0.75, MATRIX_LR=0.05)
- Memory wall proof: trained at 29.7 GB VRAM — impossible on RTX 4090 (24 GB hard limit)
- head_dim 48 crashes — another bf16 precision boundary
See round4_report.md for complete data.
- 60+ experiments — 6 LR sweeps, regularization, Adam betas, batch size, interaction effects, extended training, memory wall v2
- Best val_bpb 1.2153 (ADAM_BETAS=(0.75, 0.97), WEIGHT_DECAY=0.08)
- 2 models trained beyond 4090's 24 GB: 29,722 MB VRAM each
- New bf16 bug: beta2 < 0.97 causes NaN — optimizer precision issue
- Interaction effects: drop-one ablation shows EMBEDDING_LR and SCALAR_LR are load-bearing (NaN when dropped)
- Extended training (10 min): stable at 1.2153, run 3/3 hit NaN — bf16 accumulation degrades over time
See round5_report.md for complete data.
ASPECT_RATIO = 40 # Round 4: wider is better (32→40)
HEAD_DIM = 64 # CRITICAL — do not revert (Round 3 ablation)
WINDOW_PATTERN = "SSSSSL" # Round 4: 6-layer sliding window
TOTAL_BATCH_SIZE = 2**15
DEPTH = 8
DEVICE_BATCH_SIZE = 16
EMBEDDING_LR = 0.8
UNEMBEDDING_LR = 0.008 # Round 5: tighter (0.012→0.008)
MATRIX_LR = 0.05 # Round 4: pulled back (0.07→0.05)
SCALAR_LR = 0.7 # Round 5: tuned up (0.6→0.7)
WEIGHT_DECAY = 0.08 # Round 5: lighter (0.12→0.08)
ADAM_BETAS = (0.75, 0.97) # Round 5: beta1 0.8→0.75, beta2 0.98→0.97
WARMUP_RATIO = 0.0
WARMDOWN_RATIO = 0.75 # Round 4: tuned (0.7→0.75). CRITICAL — do not revert.
FINAL_LR_FRAC = 0.07# Install TheROCk nightly PyTorch for gfx1151
pip install torch --index-url https://rocm.nightlies.amd.com/v2/gfx1151/
# REQUIRED environment
export TORCH_ROCM_AOTRITON_ENABLE_EXPERIMENTAL=1 # 19x flash attention speedup
unset PYTORCH_HIP_ALLOC_CONF # default value crashes torch
# Clone and run
git clone https://github.com/bkpaine1/amdsense.git
cd amdsense
pip install tiktoken huggingface_hub pyarrow rustbpe
python3 prepare.py
python3 train.py# Standard PyTorch
pip install torch
# Clone and run
git clone https://github.com/bkpaine1/amdsense.git
cd amdsense
pip install tiktoken huggingface_hub pyarrow rustbpe kernels
python3 prepare.py
python3 train.pyPoint Claude Code, Cursor, or any SSH-capable coding agent at the repo:
Read program.md and kick off a new experiment run.
Expect ~12 experiments/hour. Leave it overnight for ~100 experiments.
train.py — model + training loop (agent modifies this)
prepare.py — data prep + eval (fixed, do not modify)
program.md — agent instructions
profile_rocm.py — AMD hardware profiler
runpod_benchmark.sh — NVIDIA comparison benchmark (one-script RunPod setup)
round3_report.md — Round 3: ablation + failure boundaries (33 experiments)
round4_report.md — Round 4: optimization sweep + memory wall (40 experiments)
round5_report.md — Round 5: deep squeeze, all hyperparams (60+ experiments)
nvidia_4090_comparison.md — Full cross-hardware comparison report
nvidia_4090_results.json — Raw NVIDIA benchmark data
results.tsv — Experiment log
autoresearch_agent5.py — Latest autonomous experiment runner (standalone)
Forked from karpathy/autoresearch (MIT). We submitted AMD support upstream (PR #136) — they preferred CUDA-only. So we made our own.
- Brent Paine (@bkpaine1) — hardware, vision, the guy who said "my iGPU can do this"
- Tectonic Obelisk (Claude Code, Opus 4.6) — autonomous agent, profiling, benchmark infrastructure, this README
MIT