feat(server): enforce tool_choice required and named-function semantics on chat completions

[inureyes]

Summary

/v1/chat/completions accepts tool_choice in all four OpenAI shapes but only none changes behaviour. required is validated and then ignored: the model is free to answer in prose and the response carries no tool call. A named function ({"type":"function","function":{"name":"f"}}) still renders every declared tool into the prompt and only filters the parsed calls afterwards, so a model that picks another tool returns an empty tool_calls array with finish_reason: "stop". The change makes required and the named form actually steer generation: the tool list handed to the template is narrowed, an instruction is injected into the prompt, the request is rejected with a 400 when it cannot be satisfied, and on tool-call formats whose wire shape is JSON the call is grammar-forced through the existing structured-output constraint.

Current behavior

• src/server/routes/chat.rs:1149 (validate_chat_tool_inputs) accepts auto, none, required and the object form, and that is the last time required is looked at.

• src/server/chat_request.rs:748 (effective_tools) strips tools only for none:

  if let Some(ref tc) = request.tool_choice && tc.is_none() { return None; }
  request.tools.as_deref()

  so a named choice still renders every tool.

• src/server/tool_calls/mod.rs:101 (build_tool_call_responses) and src/server/routes/chat.rs:1026 (the streaming finish path) drop parsed calls whose name differs from the named function, after generation. Nothing prevents the model from choosing another function or from not calling at all.

• src/server/types/request.rs:66 (ToolChoice) has no notion of required beyond the string.

• The Anthropic translator (src/server/anthropic_translator.rs:596, convert_tool_choice) and the Responses translator (src/server/responses_translator.rs:198) both forward into the same field, so they inherit the same gap ({"type":"any"} and {"type":"tool","name":...} are accepted and not enforced).

• The grammar engine already exists: src/server/structured.rs:535 (build_json_schema_constraint) compiles a JSON schema into an llguidance matcher and ServerGenerateOptions.structured (src/server/config.rs:187) carries it to the scheduler, which applies it per step at src/server/batch/scheduler.rs:3141.

Expected behavior

Semantics, resolved once per request before template rendering:

tool_choice	tools handed to the template	prompt instruction	constraint	error cases
absent / "auto"	all	none	none	none
"none"	none (current behaviour)	none	none	none
"required"	all	"You must call one or more of the available functions to answer the user's request. Do not answer directly without calling a function."	forced tool call when the resolved format is grammar-capable	400 invalid_request_error when tools is absent or empty
{"type":"function","function":{"name":"f"}}	only f	"You must call the 'f' function to answer the user's request. Do not call any other function and do not answer directly."	forced call to f when grammar-capable	400 when tools is empty or no tool named f exists; 400 when type is not function or name is empty

Instruction placement: appended to the first system message as "\n\n" + instruction when one exists; otherwise appended to the last user message whose content is plain text; otherwise inserted as a new leading system message. The original request object is not mutated for response echoing; the injection happens on the message list handed to the renderer.

Grammar-capable formats are the ones whose emitted call is a JSON object with a fixed wrapper and whose wrapper can be read off the loaded chat template before generation: ToolCallFormat::Hermes when the template text contains <tool_call> (Qwen2.5 / Qwen3 / Hermes templates), ToolCallFormat::MistralNemo when it contains [TOOL_CALLS], ToolCallFormat::Llama3 when it contains <|python_tag|> or renders "parameters":. There is no operator parser flag today (src/server/chat_template.rs:297, resolve_tool_call_format, only infers ATEM from template identity), so this template sniffing is the selection mechanism. For those, when the request carries no response_format, the server builds a constraint from the tool schemas:

schema(f) = {"type":"object",
             "properties": {"name": {"const": f.name},
                            "arguments": f.parameters or {"type":"object"}},
             "required": ["name","arguments"],
             "additionalProperties": false}
required: anyOf over schema(f) for every declared f
named:    schema(f) for the named f only

wrapped in the format's delimiters with an llguidance Lark grammar:

start: "<tool_call>" %json { <schema> } "</tool_call>"          # Hermes
start: %json { <schema> }                                         # Llama3
start: "[TOOL_CALLS] [" %json { <schema> } "]"                    # MistralNemo

(Llama3 uses the key parameters instead of arguments in its JSON; the schema uses the format's key.) Formats without a JSON wire shape (ATEM, Gemma 4, FunctionGemma, Command R, Functionary, Qwen3-Coder XML, MiniMax, Kimi K2, Pythonic) get the instruction and tool narrowing only. When the constrained generation finishes, finish_reason is tool_calls.

A request that combines tool_choice: "required" or a named function with response_format: json_schema is rejected with a 400 (two constraints cannot be composed).

Implementation plan

1. src/server/types/request.rs: add to impl ToolChoice:
   • pub fn is_required(&self) -> bool (Mode("required")).
   • pub fn validate(&self, tools: Option<&[Tool]>) -> Result<(), String> implementing the 400 cases of the table above (unknown mode, required without tools, named function not declared, type != "function", empty name). Move the string-mode check out of validate_chat_tool_inputs into this method so the disaggregated router front (src/server/router_front.rs) and routes/chat.rs share it.
2. src/server/chat_request.rs:
   • Change effective_tools(request) to return only the named tool for the named form (keep None for none, all tools otherwise). Because chat_carries_loop_amplifier (src/server/request_options.rs:133) reads the same helper, update its doc comment: a single narrowed tool still counts as "tools rendered".
   • Add pub(crate) fn tool_choice_instruction(choice: &ToolChoice) -> Option<String> returning the two instruction strings.
   • Add pub(crate) fn inject_tool_choice_instruction(messages: &mut Vec<Message>, instruction: &str) with the placement rule above (Message.content is MessageContent::Text for the plain-text cases; a Parts user message gets a trailing ContentPart::Text).
   • Call both from prepare_chat_request_with_cache on a cloned message list before render_*, so the prompt-cache key (template_sig / history_prompt) is derived from the prompt that is actually rendered.
3. src/server/chat_template.rs: add pub fn forced_tool_call_format(&self) -> Option<ToolCallFormat> next to default_tool_call_format (src/server/chat_template.rs:293), returning Hermes / MistralNemo / Llama3 from the template-text markers above and None otherwise (ATEM templates return None here even though default_tool_call_format reports Atem). Cache the answer in the processor at construction, like supports_tools.
   src/server/tool_calls/mod.rs: add pub fn tool_choice_grammar(format: ToolCallFormat, tools: &[Tool], choice: &ToolChoice) -> Option<String> that returns the Lark grammar text for the three grammar-capable formats and None otherwise.
4. src/server/structured.rs: add pub fn build_lark_constraint(tokenizer, lark: &str) -> Result<Arc<Mutex<StructuredOutputConstraint>>, StructuredOutputError> next to build_json_schema_constraint, using llguidance::api::TopLevelGrammar::from_lark(lark) with the same ParserLimits, the same TOK_ENV_CACHE, and the same size cap (MAX_SCHEMA_BYTES) applied to the serialized tool schemas.
5. src/server/routes/chat.rs: in both handlers (non-streaming at ~line 311 where structured is built, streaming at ~line 741), after validate: if tool_choice is required or named and the resolved format is grammar-capable and request.response_format is absent, build the constraint through build_lark_constraint on the blocking pool exactly like the response_format path and store it in options.structured. Reject the response_format + forced-tool combination with 400 before any work.
6. src/server/routes/chat.rs finish paths (non-streaming parse_tools block and streaming finish_events block around line 1020): when tool_choice is required or named and the parse produced no call, log at warn with the format name and leave finish_reason as the model produced (the instruction-only formats cannot guarantee a call). Keep the existing name filter for the named form as a last line of defence.
7. src/server/anthropic_translator.rs::convert_tool_choice: map {"type":"any"} to ToolChoice::Mode("required") and {"type":"tool","name":"f"} to ToolChoice::Specific, so the Anthropic route gets the same enforcement. src/server/responses_translator.rs already forwards tool_choice unchanged.
8. docs/responses-api.md and the server README section on tool calling: document the four modes, the grammar-capable format list, and the 400 cases.

Validation

(a) Unit tests:

• src/server/chat_request_tests.rs: effective_tools_narrows_to_named_function, tool_choice_instruction_appends_to_system_message, tool_choice_instruction_appends_to_last_text_user_message, tool_choice_instruction_inserts_system_when_no_system_and_no_text_user, named_tool_choice_changes_rendered_prompt_and_template_sig.
• src/server/types/request.rs tests: tool_choice_required_without_tools_is_rejected, tool_choice_unknown_function_is_rejected, tool_choice_object_without_function_type_is_rejected.
• src/server/tool_calls/mod.rs tests: tool_choice_grammar_hermes_required_is_anyof_over_all_tools, tool_choice_grammar_named_pins_name_const, tool_choice_grammar_is_none_for_atem_and_gemma4.
• src/server/chat_template.rs tests: forced_tool_call_format_detects_hermes_marker, forced_tool_call_format_is_none_for_atem_template.
• src/server/structured_tests.rs: lark_constraint_forces_hermes_wrapper (feed the matcher <tool_call>{"name":"get_weather","arguments":{}}</tool_call> token by token via consume_token, assert every step is allowed by compute_mask and the matcher reports stopped at the end; assert a plain-text first token is masked out).
• src/server/anthropic_translator.rs tests: anthropic_tool_choice_any_maps_to_required, anthropic_tool_choice_tool_maps_to_named.

(b) Real checkpoint: mlx-community/Qwen3-4B-4bit (Hermes tool format, grammar-capable).

./target/release/mlxcel-server -m models/qwen3-4b-4bit --port 8080
# required
curl -s localhost:8080/v1/chat/completions -H 'content-type: application/json' -d '{
  "model":"q","messages":[{"role":"user","content":"Say hello."}],
  "tool_choice":"required",
  "tools":[{"type":"function","function":{"name":"get_time","parameters":{"type":"object","properties":{"tz":{"type":"string"}},"required":["tz"]}}}]
}' | jq '.choices[0] | {finish_reason, tool_calls: .message.tool_calls}'
# named, with two tools declared
curl -s localhost:8080/v1/chat/completions -H 'content-type: application/json' -d '{
  "model":"q","messages":[{"role":"user","content":"What time is it in Seoul?"}],
  "tool_choice":{"type":"function","function":{"name":"get_weather"}},
  "tools":[{"type":"function","function":{"name":"get_time","parameters":{"type":"object","properties":{"tz":{"type":"string"}}}}},
           {"type":"function","function":{"name":"get_weather","parameters":{"type":"object","properties":{"city":{"type":"string"}}}}}]
}' | jq '.choices[0] | {finish_reason, name: .message.tool_calls[0].function.name}'

Acceptance: the first call returns finish_reason: "tool_calls" with a get_time call on 10 of 10 runs at temperature: 0.7 (the grammar makes this deterministic, not probabilistic); the second returns a get_weather call and never get_time on 10 of 10 runs; tool_choice: "required" with "tools": [] returns HTTP 400. Repeat the required case with stream: true and confirm the tool-call deltas arrive and the final chunk carries finish_reason: "tool_calls". Run the same two requests against mlx-community/Muse-Glimmer-30B-4bit (ATEM, instruction-only path) and confirm the named form yields a get_weather call on at least 9 of 10 runs.

Acceptance criteria

• tool_choice: "required" with no tools returns 400; a named function not in tools returns 400.
• The named form renders only the named tool into the prompt and changes template_sig.
• Both forced forms inject the instruction with the placement rule above.
• Templates carrying <tool_call>, [TOOL_CALLS] or <|python_tag|> grammar-force the call; other templates get instruction and narrowing only; response_format plus a forced tool is a 400.
• Anthropic tool_choice: {"type":"any"} and {"type":"tool"} map to the same enforcement.
• Unit tests listed above added and passing; real-checkpoint runs above pass.
• cargo test --workspace --profile test-fast --features metal,accelerate passes
• cargo clippy --workspace --all-targets -- -D warnings and cargo fmt --all -- --check pass

Out of scope

• parallel_tool_calls: false enforcement.
• Grammar forcing for XML and ATEM tool formats (would need per-format Lark grammars with string escaping rules).
• /v1/completions (no tool concept).