Skip to content

fix: convert_hf_to_gguf - use existing local chat_template if mistral-format model has one. #17749

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Dec 4, 2025
+16 −4
Merged

fix: convert_hf_to_gguf - use existing local chat_template if mistral-format model has one. #17749

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
ee4d781
conversion: use existing local chat_template.jinja file if mistral-fo…
SmartestWashingMachine Dec 4, 2025
64ee00b
fix --mistral-format mistakenly assuming some <=v7 chat template name…
SmartestWashingMachine Dec 4, 2025
cd40efe
Update convert_hf_to_gguf.py - change from exists() to is_file()
SmartestWashingMachine Dec 4, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 16 additions & 4 deletions convert_hf_to_gguf.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2341,19 +2341,31 @@ def _set_vocab_mistral(self):
self.gguf_writer.add_add_bos_token(True)
self.gguf_writer.add_add_eos_token(False)

template_dir = Path(__file__).parent / "models/templates/"
local_template_file_path = self.dir_model / "chat_template.jinja"

if self.is_mistral_format and local_template_file_path.is_file():
# Ministral-3 and other new Mistral models come with chat templates.
# ref: https://huggingface.co/mistralai/Ministral-3-14B-Instruct-2512/tree/main
logger.info("Using an existing Mistral local chat template.")

with open(local_template_file_path, "r", encoding="utf-8") as f:
template = f.read()
elif not self.is_mistral_format or not self.disable_mistral_community_chat_template:
template_dir = Path(__file__).parent / "models/templates/"

if not self.is_mistral_format or not self.disable_mistral_community_chat_template:
# Log only for Mistral format that the official tokenization and detokenization is via `mistral-common`.
if self.is_mistral_format:
logger.info(
"Using a Mistral community chat template. These templates can be subject to errors in early days or weeks after a release. "
"Mistral recommends to use `mistral-common` to perform tokenization and detokenization."
)
template = MistralModel.get_community_chat_template(vocab, template_dir, self.is_mistral_format)
self.gguf_writer.add_chat_template(template)
else:
logger.info("Not using a Mistral community chat template. Ensure to perform the tokenization and detokenization via `mistral-common`.")
logger.info("Not using a Mistral local or community chat template. Ensure to perform the tokenization and detokenization via `mistral-common`.")
template = None

if template is not None:
self.gguf_writer.add_chat_template(template)

def set_vocab(self):
if self.is_mistral_format:
Expand Down