Skip to content

End-to-End Token Prediction on WebGPU

Pre-release
Pre-release

Choose a tag to compare

View all tags
@qwatts-dev qwatts-dev released this 21 Feb 23:32
v0.6.0
e8e1aa2
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

This release completes the first full inference path: Embed → SwiGLU MLP → RMSNorm → LM Head → Argmax → Decode, enabling real word prediction directly in browser WebGPU.

Highlights

  • Added dense FP32 LM Head compute path and vocab-logit decoding flow.
  • Added LM Head extraction pipeline (tied embeddings) and generated sparse_lm_head.bin (FP16 vocab slice).
  • Fixed WebGPU storage-buffer limit handling by requesting adapter hardware limits (unblocks 160 MB LM Head buffer).
  • Added RMSNorm stabilization before LM Head to prevent FP32 overflow and NaN/zero logits.
  • Verified cross-device deterministic output (M2 Max, M3 iPad, A16 iPhone): identical predicted token (" volume", ID 8286, logit 207.9285).
  • Updated docs with mobile/ngrok testing flow and benchmark results for v0.6.0.
Assets 2
Loading