Skip to content

fix mistral3 vllm backend ignore consolidated.safetensors#7074

Merged
hjh0119 merged 2 commits intomodelscope:mainfrom
hjh0119:ministral3
Dec 16, 2025
Merged

fix mistral3 vllm backend ignore consolidated.safetensors#7074
hjh0119 merged 2 commits intomodelscope:mainfrom
hjh0119:ministral3

Conversation

@hjh0119
Copy link
Copy Markdown
Collaborator

@hjh0119 hjh0119 commented Dec 16, 2025

test with vllm 0.12.0

swift infer \
    --model mistralai/Ministral-3-3B-Instruct-2512 \
    --infer_backend vllm \
    --vllm_engine_kwargs '{"config_format": "mistral", "load_format": "mistral", "tokenizer_mode": "mistral"}'

@gemini-code-assist
Copy link
Copy Markdown
Contributor

Summary of Changes

Hello @hjh0119, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request resolves a critical loading issue for Mistral 3 models when utilizing the vLLM inference backend. Previously, the backend would overlook essential consolidated.safetensors files, leading to model loading failures. The changes ensure that the model registration process correctly accounts for all required files, thereby enabling seamless integration and functionality with vLLM.

Highlights

  • VLLM Backend Fix: Addressed an issue where the vLLM backend was incorrectly ignoring consolidated.safetensors files for Mistral 3 models, preventing proper model loading.
  • Model Registration Update: Modified the register_model calls for ModelArch.llava_hf in swift/llm/model/model/mistral.py to explicitly include ignore_patterns=[], ensuring all necessary model files are considered during loading.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds an ignore_patterns parameter to the mistral_2512 and mistral_2512_thinking model registrations to address an issue with the vLLM backend. My review points out a potential discrepancy between the pull request's title, which suggests ignoring consolidated.safetensors, and the code change, which sets an empty ignore list. I've provided suggestions to correct this potential issue.

model_arch=ModelArch.llava_hf,
requires=['transformers>=5.0.0.dev0', 'mistral-common>=1.8.6'],
tags=['vision'],
ignore_patterns=[],
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The pull request title suggests that consolidated.safetensors should be ignored. However, setting ignore_patterns=[] will result in no files being ignored, which seems to contradict the stated goal. If the intention is to ignore consolidated.safetensors, this should be ignore_patterns=['*consolidated.safetensors'] to match files with that name.

Suggested change
ignore_patterns=[],
ignore_patterns=['*consolidated.safetensors'],

model_arch=ModelArch.llava_hf,
requires=['transformers>=5.0.0.dev0', 'mistral-common>=1.8.6'],
tags=['vision'],
ignore_patterns=[],
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Similar to the comment above, this change seems to contradict the intent described in the pull request title. If consolidated.safetensors should be ignored, this should be ignore_patterns=['*consolidated.safetensors'].

Suggested change
ignore_patterns=[],
ignore_patterns=['*consolidated.safetensors'],

@hjh0119 hjh0119 merged commit ba732db into modelscope:main Dec 16, 2025
2 of 3 checks passed
@hjh0119 hjh0119 deleted the ministral3 branch December 16, 2025 09:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants