Skip to content

v1.3.1 — Fix ONNX Runtime crash with q4 dtype

Latest

Choose a tag to compare

@xezpeleta xezpeleta released this 27 Jun 12:12

What's Changed

  • Default dtype changed from q4f16 to q4 — fixes ONNX Runtime crash in browsers.

The Bug

q4f16 uses fp16 activations, which trigger a known graph optimization bug in ONNX Runtime WASM:

SimplifiedLayerNormFusion / InsertedPrecisionFreeCast

This happens across all fp16 models in browsers, not just Qwen2.5.

The Fix

q4 uses fp32 activations — fully compatible with ONNX Runtime WASM, no crashes.

Tradeoff

dtype Size Compatible
q4 (new default) ~750MB ✅ Browser + Node
q4f16 (old) ~500MB ❌ Crashes in browser
fp16 ~1GB ❌ Crashes in browser

References