Repository navigation
v0.5.1 - agent-serving robustness
Patch release: agent-serving robustness for local coding agents (opencode, codex) and multi-model deployments.
- Fixed: fatal bf16 fused-SwiGLU Metal JIT crash that took the whole server down under Gemma 4 12B / any bf16 model on the fused FFN (#188).
- stream_options.include_usage honored — coding agents now show real context/token counts instead of zero (#187).
- config.toml reads serving knobs (max_loaded_models, num_parallel, keep_alive, max_queue) — set
max_loaded_models = 2to keep an embedding model and a generation model resident on one port, routed by the request'smodelfield (#189).
Install/upgrade: brew upgrade krillm (or brew tap srvsngh99/krillm && brew install krillm). Full notes: CHANGELOG.md (0.5.1).