Repository navigation
v1.0.1 — ANE default restored
Hotfix for v1.0.0. The default compute units for Qwen3.5 had been temporarily switched to GPU during profile/benchmark debugging and weren't restored before the ship commit. v1.0.0 ran on GPU by default, consuming ~3 GB of extra Metal heap beyond the model weights.
Fix
Default restored to .cpuAndNeuralEngine for both prefill and decode. Behaves as the v1.0.0 release notes originally described.
Memory note (applies to both ANE and GPU paths)
A 0.8B fp16 model inherently needs ~1.4 GB of RAM just for weights — this is unified memory on Apple Silicon, shared between CPU mmap and the ANE plan cache. Total app memory on ANE mode sits around 1.6-2 GB (weights + ~200 MB ANE runtime + ~300 MB app baseline).
"0 GB Metal heap" means the GPU is not allocating additional buffer memory. On GPU mode, Metal allocates another ~3 GB on top of the 1.6 GB baseline, bringing total to ~4.6 GB.
| path | Metal heap | total app memory |
|---|---|---|
| ANE (default) | 0 GB | ~1.6 GB |
| GPU (bit-exact) | ~3 GB | ~4.6 GB |
| CPU | 0 GB | ~1.6 GB |
Included
- #116 fix(qwen3.5): restore ANE default compute units