What's Changed
- Default dtype changed from
q4f16toq4— fixes ONNX Runtime crash in browsers.
The Bug
q4f16 uses fp16 activations, which trigger a known graph optimization bug in ONNX Runtime WASM:
SimplifiedLayerNormFusion / InsertedPrecisionFreeCast
This happens across all fp16 models in browsers, not just Qwen2.5.
The Fix
q4 uses fp32 activations — fully compatible with ONNX Runtime WASM, no crashes.
Tradeoff
| dtype | Size | Compatible |
|---|---|---|
| q4 (new default) | ~750MB | ✅ Browser + Node |
| q4f16 (old) | ~500MB | ❌ Crashes in browser |
| fp16 | ~1GB | ❌ Crashes in browser |