Skip to content

AbstractCore v2.16.1

Choose a tag to compare

@github-actions github-actions released this 26 Sep 21:13
· 478 commits to main since this release

Local MLX models now see the conversation exactly as their own chat template renders it, so agent
loops with tool calls keep working on Qwen3.x MLX builds with thinking on, and every MLX response says
which prompt renderer it used.

Fixed

  • MLX prompts are rendered by the model's own chat template (tokenizer.apply_chat_template) on every
    MLX lane (mlx-lm, native MTP/APC, vision add-on). Earlier assistant turns keep their tool calls, tool
    results are rendered as the template renders them (Qwen3.x: <tool_response> inside a user turn),
    tools= reaches the template's own tool block, and the generation prompt opens <think> when the
    template does. Agent loops on Qwen3.x MLX builds with thinking on used to stop at the third
    iteration with a one-sentence announcement ("Let me verify … before writing the digest.") and no
    tool call; they now keep calling tools and finish with an answer. A system message in the middle of
    a conversation is sent as a <system_instruction> user turn, as on strict OpenAI-compatible servers.
    Models whose tokenizer has no chat template keep the built-in renderer; the log names the renderer
    once per model. Prompt-cache fragments rendered by a chat template carry a new serializer version
    (mlx-prompt-fragment/v2:chat-template:<sha>), so KV artifacts compiled with the previous renderer
    are rebuilt rather than reused.
  • An MLX conversation whose history holds a tool call with non-JSON-object arguments stays on the
    chat-template renderer: the arguments reach the template as {"raw_arguments": "<text>"} instead of
    making every later call of the run fall back to the built-in renderer. Every MLX response names its
    renderer in metadata["prompt_renderer"] (chat_template, builtin or builtin_fallback, with
    metadata["prompt_renderer_reason"] for a fallback). A streamed call carries both keys on its last
    chunk, including when the stream carries reasoning.