v1.1.0 — Qwen3.5 2B on iPhone ANE

What's new

First 2B-class hybrid SSM + attention LLM shipped on CoreML.

Qwen3.5 2B on iPhone ANE

• Qwen3.5 2B on iPhone ANE — 24-layer hybrid SSM + attention (2.04 B params) split into 4 INT8 transformer chunks (6 layers each). Every chunk compiles on iPhone 17 Pro ANE at ≥ 90% op placement. ~17 tok/s decode, 2048-token context. Bundle: mlboydaisuke/qwen3.5-2B-CoreML.
• mmap'd fp16 embed sidecar — embed_tokens.weight ships as a raw 1 GB fp16 file (embed_weight.bin), Swift mmap(..., MAP_PRIVATE) with MADV_RANDOM. Only the handful of rows touched per prompt page in, and those pages stay "clean" so they don't count against phys_footprint. Reported memory during generation: ~200 MB (vs ~2 GB with a CoreML chunk_embed mlpackage that would dequantize the full 1 GB into process memory).
• Why 4 chunks (not 2) — palettization shrinks disk, not ANE memory: INT8 weights re-expand to fp16 inside the ANE region, and iPhone ANE's per-mlprogram compile envelope rejected the 2-chunk split (~2 GB fp16/chunk) with MILCompilerForANE: Couldn't communicate with a helper application → silent GPU fallback at 3.4 GB Metal heap and 7 tok/s. 4 chunks at ≤ 1 GB INT8 (≤ ~1.7 GB fp16) each match the Gemma 4 E4B envelope that's proven ANE-resident.
• 2048-token context window — bumped from 128 so chat turns don't truncate after ~10 lines. Only full-attention state scales with max_seq (6 layers × 2 × max_seq × 256 × fp16 × 2); +22 MB total vs the 128-token ceiling.
• App binary shrunk ~5 GB — removed stale Qwen3.5-0.8B mlpackages from the Xcode target.
• ChatView tap-to-dismiss — tap outside the input field or drag-scroll to dismiss the keyboard.

Device spec (iPhone 17 Pro)

metric	value
Decode	17 tok/s
phys_footprint (inference)	~200 MB
Metal heap (sustained)	0 GB
ANE placement (chunk_a/b/c/d)	90.7% / 91.1% / 90.7% / 90.8%
Context window	2048 tokens
Bundle (HF)	2.4 GB

Full changelog: docs/RELEASE_v1_1_0.md.