Skip to content

MLX Runner v1.1.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 31 Oct 17:32
· 2 commits to main since this release
v1.1.0
e61f62b
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Tool calling in the local server (OpenAI-compatible):

  • Function tools in Chat Completions (tools, tool_choice, parallel_tool_calls, assistant tool_calls, tool messages) and the Responses API (function tools, function_call / function_call_output items, function_call output items with their streaming events), streaming and non-streaming.
  • Prompts use each model's native tool format: Hermes (Qwen2.5, Qwen3), Llama 3.1+ and Mistral v3. They match the official chat templates and mistral-common byte for byte, which tests enforce.
  • tool_choice: auto, none, required (the function name is constrained to the defined tools during decoding) and a named function.
  • Requests are validated: tool count and names, JSON object arguments, tool results that must answer an earlier call, and a nesting limit. Models without a tool format (Phi-3.5) answer tool requests with a 400.
  • Tested end to end with the official openai Python SDK on Qwen3 0.6B/8B, Qwen2.5 0.5B, Llama 3.2 1B and Mistral 7B v0.3.

Downloads, checksums and verification: https://mlx-runner.21no.de/#verify

Assets 6
Loading