Repository navigation
AbstractCore v2.16.1
Local MLX models now see the conversation exactly as their own chat template renders it, so agent
loops with tool calls keep working on Qwen3.x MLX builds with thinking on, and every MLX response says
which prompt renderer it used.
Fixed
- MLX prompts are rendered by the model's own chat template (
tokenizer.apply_chat_template) on every
MLX lane (mlx-lm, native MTP/APC, vision add-on). Earlier assistant turns keep their tool calls, tool
results are rendered as the template renders them (Qwen3.x:<tool_response>inside a user turn),
tools=reaches the template's own tool block, and the generation prompt opens<think>when the
template does. Agent loops on Qwen3.x MLX builds with thinking on used to stop at the third
iteration with a one-sentence announcement ("Let me verify … before writing the digest.") and no
tool call; they now keep calling tools and finish with an answer. A system message in the middle of
a conversation is sent as a<system_instruction>user turn, as on strict OpenAI-compatible servers.
Models whose tokenizer has no chat template keep the built-in renderer; the log names the renderer
once per model. Prompt-cache fragments rendered by a chat template carry a new serializer version
(mlx-prompt-fragment/v2:chat-template:<sha>), so KV artifacts compiled with the previous renderer
are rebuilt rather than reused. - An MLX conversation whose history holds a tool call with non-JSON-object
argumentsstays on the
chat-template renderer: the arguments reach the template as{"raw_arguments": "<text>"}instead of
making every later call of the run fall back to the built-in renderer. Every MLX response names its
renderer inmetadata["prompt_renderer"](chat_template,builtinorbuiltin_fallback, with
metadata["prompt_renderer_reason"]for a fallback). A streamed call carries both keys on its last
chunk, including when the stream carries reasoning.