Skip to content

v0.12.0 — Fused SDPA attention (opt-in)

Choose a tag to compare

@cattolatte cattolatte released this 07 Jul 01:25
3a6ce4e

Optional attention="sdpa" config uses PyTorch's fused scaled_dot_product_attention (RoPE + KV-cache + causal preserved), numerically equivalent to the eager path and ~1.28× faster on the 10.7M model. Default stays eager; old checkpoints load unchanged.