Skip to content

v0.5.1 - agent-serving robustness

Choose a tag to compare

@srvsngh99 srvsngh99 released this 09 Jun 09:46
· 221 commits to main since this release

Patch release: agent-serving robustness for local coding agents (opencode, codex) and multi-model deployments.

  • Fixed: fatal bf16 fused-SwiGLU Metal JIT crash that took the whole server down under Gemma 4 12B / any bf16 model on the fused FFN (#188).
  • stream_options.include_usage honored — coding agents now show real context/token counts instead of zero (#187).
  • config.toml reads serving knobs (max_loaded_models, num_parallel, keep_alive, max_queue) — set max_loaded_models = 2 to keep an embedding model and a generation model resident on one port, routed by the request's model field (#189).

Install/upgrade: brew upgrade krillm (or brew tap srvsngh99/krillm && brew install krillm). Full notes: CHANGELOG.md (0.5.1).