v0.2.0: Dropout, Profiling, CUDA Auto-Tune

What's New

Dropout

• Configurable dropout ("dropout": 0.2 in config)
• Disabled during eval, zero overhead when rate=0
• Enabled the Shakespeare nanoGPT benchmark comparison (best val 1.55)

Profiling

• -cpuprofile and -memprofile flags with zero overhead when disabled
• Remote GPU profiling workflow documented (signed URL upload from RunPod)

CUDA Performance

• Auto-tunes MLX CUDA graph batch size from model IR op count
• Reduces kernel dispatch overhead on small models
• Large models saturate GPU regardless (verified 100% on A40 at 125M params)

New IR Ops

• GatherPositions: [B,T,D] + positions[K] → [B,K,D]
• ScatterPositions: [B,T,D] + updates[B,K,D] + positions[K] → [B,T,D]
• General-purpose position-indexing primitives for sparse/selective computation

Architecture

• skip_attention option for plain blocks (FFN-only variants)
• Config validator checks block registry for external block types
• Optimizer group building moved from train to gpu (SoC cleanup)

Infrastructure

• GitHub Actions CI with golangci-lint v2
• PyTorch numerical parity test (exact to 8 decimal places)
• GPU integration tests for all 12 example configs
• Shakespeare benchmark with Metal and CUDA results
• Docker: curl/wget in RunPod image, handler subprocess fix
• Homebrew formula at mrothroc/tap/mixlab

Install

brew install mrothroc/tap/mixlab

Or build from source: make build