Apple Foundation Models vs local GGUF — when to use which?

[sha367]

Both run fully on device, but the trade-offs differ:

Foundation Models (Apple Intelligence)

• ✅ Zero download, OS-managed, NPU-fast
• ✅ Free — no tokens, no infra
• ❌ iOS/macOS 26+ and recent hardware only
• ❌ One model, no choice, guardrails you don't control

Local GGUF (llama.cpp etc.)

• ✅ Any model you want, any platform
• ✅ Full control over sampling/prompts
• ❌ 0.8–2 GB download per model
• ❌ CPU/GPU inference — hotter and slower than NPU

We ship both in privateSLM (system model by default, GGUF as fallback/override). Curious how others decide — do you let users pick, or auto-select?