Features

Features

TinyTitan is a native Swift 6 and Metal runtime for compatible Qwen-based MoE text models on Apple Silicon, from 35B-A3B up to Qwen3.8-Flash-Next 125B-A6B.

Core capabilities

Area	What TinyTitan provides
Models	Verified 4-bit and 8-bit Ornith 1.5 35B-A3B, Qwen 3.6 35B-A3B, Qwen-AgentWorld 35B-A3B and KAT-Coder-V2.5-Dev 35B-A3B, plus 4-bit and 8-bit Qwen3.8-Flash-Next 125B-A6B and the dense Qwen 3.5 2B/4B/9B on either engine; text-only Ornith 8-bit is the default and 6-bit no longer loads
Memory	Bounded SSD streaming for routed experts, with hit/fixup overlap between cache-hit execution and miss reads
Performance	Tiled three-stage GPU Top-K sampling; Neural Engine prefill on by default, 2.31x on long prompts, falling back to the GPU when no sidecar is present; per-family expert-cache sizing and speculative expert prefetch
Interfaces	Command-line generation, installer/verifier, and a loopback OpenAI-compatible server
Context	Native 262K or optional 512K/1M YaRN, with growing 16-, 8-, or 4-bit KV state (8-bit default)
Generation	Temperature, Top-K, Top-P, repetition penalty, seed, stop strings, concise mode, and a thinking switch that is off/on everywhere and off/low/medium/xhigh on Qwen3.8-Flash-Next
Prompt reuse	Exact live and multi-prefix state reuse, with an optional private SSD cache
API	Chat Completions and Responses APIs, JSON/SSE streaming, and function-tool calls
Coding clients	Launch helpers for Codex, Claude Code, Qwen Code, OpenCode and Zed; a chat-only fast alias reduces agent-prompt prefill
Residency	Optional lazy model loading, idle unload, and manual unload

Important limits

• Text and function calls only; Ornith vision and audio input are not included.
• One loaded model and one generated choice per server.
• The local server has no remote authentication or TLS.
• Tool calls are returned to the client; TinyTitan never executes or authorizes tools itself.
• Native one-layer MTP speculative decoding is optional for Qwen and Ornith, target-verified, and experimental. It cannot be combined with YaRN. Current M3 measurements found no Ornith speed benefit, so it remains off by default.
• Thinking is a chat-template switch: off or on for Ornith, Qwen 3.6, AgentWorld and KAT-Coder, and off / low / medium / xhigh for Qwen3.8-Flash-Next, which carries real effort levels rather than a binary.

Start with Getting Started, then use Runtime Controls only when the defaults do not fit the workload.