TinyGPT v1.0.1 — refactor & cleanup

Maintenance release. Internal refactoring and dead-code removal only — no user-facing behavior changes.

Duplication removed

• Swift CLI: 22 copy-pasted helper definitions consolidated into 4 shared helpers in `CLIHelpers.swift` (`formatLargeInt`, `argmax`, `mlComputeUnits`, `runBlocking`).
• evals: shared `evals/_common.sh` (`resolve_tinygpt` + `fail`) replaces the binary-resolution boilerplate copy-pasted across 4 smoke scripts; added `evals/common-smoke.sh` test.

Dead weight removed

• 15 zero-reference one-off browser dev scripts (audit/measure/screenshot/inspect).
• `browser/_legacy_html/` + `convert_to_astro.mjs` (retired Vite→Astro regeneration pipeline).

Housekeeping

• Archived 21 superseded Python iteration scripts to `scripts/archive/` (history preserved via git mv); repointed 9 docs.

Verification

swift build --build-tests, browser tsc + astro build (299 pages), and all eval smokes pass green.

TinyGPT v1.0 — public release

A GPT-2-shaped transformer, written from scratch and trained in your browser tab. Python reference, hand-written C++/WASM, hand-written WGSL — the same model at three levels, every gradient pinned down by a test.

Headline measured numbers (Apple M-series, same seed, same data)

Backend	Step time	Loss	Drift
WASM SIMD (baseline)	6.8 s	2.9385	—
WebGPU + blocked4 + FA2 fwd + FA2 bwd	0.7 s	2.8650	2.5%

~9.7× end-to-end speedup, 2.5% loss drift — float-reorder noise only, the model trains identically.

Major levers shipped

• Memory64 lifts the 4 GB tab heap → 473M-param model allocates
• Blocked4 matmul (workgroup-shared tiling + 4×4 register block) — default forward + backward
• vec4-aligned global loads on top of blocked4
• Subgroup-cooperative layernorm + cross-entropy reductions
• Flash Attention 2 (forward + backward) — backward recomputes attention from a saved log-sum-exp, which let the forward drop its O(B·H·T²) attention writeback entirely. ~67 MB of memory traffic per layer per step saved at Mega-class shapes.
• Watch-the-model-think view — click any generated token → see top-K next-byte probabilities and per-head attention thumbnails
• PostHog analytics, three events, DNT-respecting, no PII
• Astro frontend, five pages: /, /roadmap, /devlog, /speedup, /webgpu-test

Honest negative results (documented in the devlog)

• f16-packed storage on top of tiled matmul — wins standalone, doesn't compound on top of tiling because the kernel becomes compute-bound
• 8×8 register block — lost to 4×4 at every size, likely register spill
• vec4 first integration attempt — wins standalone but a silent WGSL access-mode mismatch broke non-square shapes; root-caused and fixed

End-to-end parity bar

tests/test_webgpu_train.mjs runs 50 WASM steps + 50 WebGPU steps on the same seed and asserts loss drift < 5%. Every kernel integration in this release went through that gate.

What's next

• Pre-trained model gallery — Cloudflare R2-hosted, manifest-driven
• Native macOS app — MLX-Swift + SwiftUI, same .tinygpt file format both ways

Docs

• browser/src/pages/devlog.astro — long-form session log with the negative results
• browser/src/pages/roadmap.astro — lever-by-lever
• docs/fa2_forward_notes.md + docs/fa2_backward_notes.md
• docs/online_softmax_in_attention.md
• docs/watch_the_model_think.md