Skip to content

[Bugfix] Use lazy string reference for DeepseekV3Config in config registry #28958

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 6 commits into
base: main
Choose a base branch
from
+8 −2
Open

[Bugfix] Use lazy string reference for DeepseekV3Config in config registry #28958

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
4a12840
[Bugfix] Fix config registry issue
yongming-qin Nov 18, 2025
d16f226
[Bugfix] Expose DeepseekV3Config in configs for lazy loading
yongming-qin Nov 18, 2025
1d531f1
[Bugfix] Break a long line to meet the 88-character limit
yongming-qin Nov 18, 2025
51ce742
Apply ruff auto-fixes
yongming-qin Nov 18, 2025
dc40137
Merge branch 'main' into bugfix-deepseekv3config-lazy
yongming-qin Nov 18, 2025
5062047
Polish the docstring
yongming-qin Nov 19, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions vllm/transformers_utils/config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
RepositoryNotFoundError,
RevisionNotFoundError,
)
from transformers import DeepseekV3Config, GenerationConfig, PretrainedConfig
from transformers import GenerationConfig, PretrainedConfig
from transformers.models.auto.image_processing_auto import get_image_processor_config
from transformers.models.auto.modeling_auto import (
MODEL_FOR_CAUSAL_LM_MAPPING_NAMES,
Expand Down Expand Up @@ -80,7 +80,7 @@ def __getitem__(self, key):
afmoe="AfmoeConfig",
chatglm="ChatGLMConfig",
deepseek_vl_v2="DeepseekVLV2Config",
deepseek_v32=DeepseekV3Config,
deepseek_v32="DeepseekV3Config",
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot Nov 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

This change to use a string reference for lazy loading is a good idea, but it will cause a runtime AttributeError as implemented.

The LazyConfigDict expects to find the config class within the vllm.transformers_utils.configs module when a string is provided. However, DeepseekV3Config is imported from transformers and is not exposed in that module after your change to remove the direct import in this file.

To fix this, you need to make DeepseekV3Config available in vllm.transformers_utils.configs. You can achieve this by adding from transformers import DeepseekV3Config to vllm/transformers_utils/configs/__init__.py and including "DeepseekV3Config" in its __all__ list. This will preserve the lazy loading behavior, as vllm.transformers_utils.configs is only imported when a config is first accessed from the registry.

Copy link
Author

@yongming-qin yongming-qin Nov 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to me. I have added from transformers import DeepseekV3Config in vllm/transformers_utils/configs/__init__.py
and added comments to explain such case.

flex_olmo="FlexOlmoConfig",
Comment on lines 81 to 84
Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot Nov 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge DeepseekV3 configs now fail to resolve

The registry now stores deepseek_v32 as the string "DeepseekV3Config", but LazyConfigDict.__getitem__ only resolves strings from vllm.transformers_utils.configs. That package does not export DeepseekV3Config, so any model with model_type deepseek_v32 will raise AttributeError when loading the config, even on transformers versions that provide the class. Previously the config loaded correctly when the class was available; after this change Deepseek V3 models cannot be parsed at all.

Useful? React with 👍 / 👎.

Copy link
Author

@yongming-qin yongming-qin Nov 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Solved in the reply to gemini-code-assist[bot]
https://github.com/vllm-project/vllm/pull/28958/files#r2539426365

kimi_linear="KimiLinearConfig",
kimi_vl="KimiVLConfig",
Expand Down
6 changes: 6 additions & 0 deletions vllm/transformers_utils/configs/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,13 @@

- There is no configuration file defined by HF Hub or Transformers library.
- There is a need to override the existing config to support vLLM.
- The HF model_type isn't recognized by the Transformers library but can
be mapped to an existing Transformers config, such as
deepseek-ai/DeepSeek-V3.2-Exp.
"""

from transformers import DeepseekV3Config

from vllm.transformers_utils.configs.afmoe import AfmoeConfig
from vllm.transformers_utils.configs.chatglm import ChatGLMConfig
from vllm.transformers_utils.configs.deepseek_vl2 import DeepseekVLV2Config
Expand Down Expand Up @@ -44,6 +49,7 @@
"AfmoeConfig",
"ChatGLMConfig",
"DeepseekVLV2Config",
"DeepseekV3Config",
"DotsOCRConfig",
"EAGLEConfig",
"FlexOlmoConfig",
Expand Down