Qwen Fixed chat templates

[troymroberts]

https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates

this guy has been working on fixing the default qwen chat templates, might be worth a look

[noonghunna]

Thanks for surfacing this — looked through the README and the seven fixes line up with several failure classes we've been seeing on our quality matrix. Three look particularly worth A/B'ing:

1. Empty <think></think> blocks accumulating in past turns — exactly the kind of context bloat that'd punish multi-turn agentic packs. Our hermesagent-20 pack is at 5% pass on the canonical dual.yml-class Qwen 3.6 27B baseline; if empty-think pollution is a root cause it'd move materially.
2. Unclosed thinking before tool call — we've reproduced this exact pattern (<think>... followed by <tool_call> without a closing </think>). vLLM's --reasoning-parser qwen3 papers over some of it but not all. Could explain part of the toolcall-15 / cli-40 verifier_fail tail.
3. </thinking> hallucination on Qwen 3.6 — yes, we've seen this. Fix detects both </think> and </thinking> as valid close tokens.

The other four (developer role, no-user-query graceful fallback, |items filter for C++ engines, type-aware tojson) are also real bugs but lower-priority for our vLLM-on-Ampere stack — the |items one matters for llama.cpp users though.

Integration plan (assuming we proceed):

• Mount qwen3.6/chat_template.jinja into the Qwen 3.6 compose
• Pass via vLLM CLI --chat-template <path> (cleaner than editing tokenizer_config.json — keeps the model dir pristine and lets us A/B by toggling the flag)
• Re-run quality-test.sh --full on a representative compose (probably dual/int8.yml since it's our matched-config anchor)
• Compare 8-pack scores against the rebench baseline, particularly hermesagent-20 / toolcall-15 / cli-40 / reasonmath-15

If the A/B moves hermesagent-20 by anything significant (≥+15%), this becomes a default-on change on our shipped Qwen 3.6 composes. If it's flat or negative on the matched-config quality matrix, we ship it as opt-in via env var.

Will queue it up. cc @froggeric — heads up that we'll be testing your templates against an 8-pack quality matrix on dual 3090 + AutoRound INT4 KV variants, happy to share numbers when they're in.

[noonghunna]

First-pass A/B is in — hermesagent-20 45% → 60% (+15pp), 7 other packs flat.

Setup:

• Compose: dual/int8.yml (Qwen 3.6 27B AutoRound INT4 + INT8 PTH KV, MTP n=4, dual 3090, 262K ctx, max-num-seqs=2)
• Override: --chat-template /path/to/froggeric/qwen3.6/chat_template.jinja mounted into the container
• Sanity-checked the template was active via /v1/chat/completions/render — developer role accepted without exception, <|think_off|> stripped from rendered prompt
• Same vLLM nightly + same commit otherwise; pre-fix baseline is from a 2026-05-10 rebench leg

Pack	Baseline (2026-05-10)	Froggeric template (today)	Δ
toolcall-15	10/15 (67%)	10/15 (67%)	0
instructfollow-15	13/15 (87%)	13/15 (87%)	0
structoutput-15	13/15 (87%)	13/15 (87%)	0
dataextract-15	15/15 (100%)	15/15 (100%)	0
reasonmath-15	6/15 (40%)	6/15 (40%)	0
bugfind-15	11/15 (73%)	11/15 (73%)	0
hermesagent-20	9/20 (45%)	12/20 (60%)	+3 (+15pp)
cli-40	17/40 (42%)	17/40 (42%)	0
TOTAL	94/150 (63%)	97/150 (65%)	+3 (+2pp)

The pack that moved (hermesagent-20) is exactly the one the README claims would benefit — multi-turn agentic loops where the three fixes (empty <think></think> past-turn pollution, no-user-query graceful fallback, unclosed thinking before tool call) compound. The other packs are mostly single-turn so no contact with those failure modes.

Caveats before declaring victory:

• Single run — won't fully separate from hermesagent-20's historical run-to-run variance until we re-bench.
• We landed a separate change (vllm#35936 overlay) earlier today on the same compose. Quality-test uses tool_choice="auto" so the overlay should be a no-op here, but a no-template control run on today's commit would isolate it cleanly. Queued.

What's next:

• Run no-template control today on the same commit, confirm the +15pp is the template (not #35936 spillover)
• If confirmed: ship as default-on for Qwen 3.6 composes via --chat-template mount, post a follow-up here
• Added froggeric/Qwen-Fixed-Chat-Templates to our docs/UPSTREAM.md watch list under a new "Community templates / model assets" section so we re-check on updates

cc @froggeric — the template moved a real multi-turn agent benchmark on dual 3090 + AutoRound INT4. Will share matched-control numbers once they land.