Unwrap `text_config` in `AutoModelFor*.from_config` by jamesbraza · Pull Request #45770 · huggingface/transformers

jamesbraza · 2026-05-04T00:39:39Z

@ArthurZucker @Cyrilvallez @zucchini-nlp

jamesbraza · 2026-05-04T00:41:06Z

+                # TODO: Validate that copying the parent quantization config to the text sub-config preserves
+                # modules_to_not_convert and skip-module matching when composite-model module prefixes differ.


Matching #45494 here, fyi

zucchini-nlp · 2026-05-04T08:46:36Z

+            if model_class.config_class == config.sub_configs.get("text_config", None):
+                # TODO: Validate that copying the parent quantization config to the text sub-config preserves
+                # modules_to_not_convert and skip-module matching when composite-model module prefixes differ.
+                parent_config = config
+                config = config.get_text_config()
+                # Propagate quantization_config from the composite parent config so that
+                # `get_hf_quantizer` can correctly detect the model as pre-quantized.
+                if hasattr(parent_config, "quantization_config"):
+                    config.quantization_config = parent_config.quantization_config
            return model_class._from_config(config, **kwargs)


tbh i am not even sure we are actually using it, except for qwen3.5. From what I see in mapping, gemma models will load a VLM class. And this makes sense if a complete config is passed/downloaded

Not sure what was the idea with qwen3.5 as I wasn't there when shipping the model, but a VLM class should be completely capable of running text-only samples

Fwiw I just re-ran the minimal reproducer from #45759 for Qwen/Qwen3.6-35B-A3B, the crash I reported does happen. This makes sense because qwen3_5_moe backs Qwen/Qwen3.6-… too.

a VLM class should be completely capable of running text-only samples

My motivation mainly is for SkyRL's FSDP setup it wants AutoModelForCausalLM to give back a text-only class. Mainly this PR just brings from_config in line with from_pretrained's current behavior.

Are you thinking something else? I think I may not be fully appreciating your comment here.

Qodo-Free-For-OSS · 2026-05-04T09:00:32Z

Hi, _BaseAutoModelClass.from_config and .from_pretrained now copy quantization_config onto the unwrapped text config whenever the parent has the attribute, even if the value is None; downstream quantizer detection treats this as pre-quantized and will crash on None.get(...).

Severity: action required | Category: correctness

How to fix: Guard quantization_config propagation

Agent prompt to fix - you can give this to your LLM of choice:

Issue description

AutoModelFor*.from_config/from_pretrained now propagates quantization_config from a composite parent config to the unwrapped text_config. If the parent has quantization_config=None, the propagated attribute presence causes get_hf_quantizer() to treat the model as pre-quantized and crash when it calls .get() on None.

Issue Context

get_hf_quantizer uses hasattr(config, "quantization_config") (not value checks) to decide pre_quantized, and AutoHfQuantizer.supports_quant_method() assumes a dict-like object and calls .get().

Fix Focus Areas

src/transformers/models/auto/auto_factory.py[241-249]

src/transformers/models/auto/auto_factory.py[394-402]

What to change

Only set config.quantization_config on the text sub-config when the parent value is not None.

Prefer getattr(parent_config, "quantization_config", None) over hasattr.

Consider also preserving an existing non-None quantization_config already present on the text sub-config (avoid overwriting).

Qodo code review - free for open-source.

Mirror the existing from_pretrained text_config unwrap into from_config so the composite-config -> text-only-model "VLM compatibility" mapping (e.g. Qwen3.5 / Qwen3.5-MoE) works for meta-init paths too. Without this, AutoModelForCausalLM.from_config(Qwen3_5MoeConfig(...)) fails with AttributeError because the text-only model class receives the composite config. Also propagate quantization_config from the parent so pre-quantized detection still works, mirroring PR huggingface#45494. Add a parametrized regression test covering both from_pretrained and from_config in the qwen3_5 and qwen3_5_moe model test files, and extend the existing gemma3 test_automodelforcausallm to match. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

The unwrap branch in AutoModelFor*.from_config never triggers for gemma3 (its MODEL_FOR_CAUSAL_LM_MAPPING entry routes Gemma3Config to the VLM class Gemma3ForConditionalGeneration, whose config_class is Gemma3Config itself, not Gemma3TextConfig). The new from_config parameterization therefore exercised only the pre-existing path and added no coverage of the fix.

`hasattr(parent_config, "quantization_config")` returns True even when the value is `None` (e.g. a config.json containing `"quantization_config": null`, or a dequantization step that cleared the attribute). The previous propagation copied that None onto the unwrapped text sub-config, and `from_pretrained`'s downstream `get_hf_quantizer` keys off `hasattr` to set `pre_quantized=True`, then crashes inside `AutoHfQuantizer.supports_quant_method(None)` on `None.get(...)`. Switch to `getattr(parent_config, "quantization_config", None) is not None` at both call sites so absent and None collapse into the same "don't propagate" case. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

github-actions · 2026-05-04T17:55:11Z

[For maintainers] Suggested jobs to run (before merge)

run-slow: auto, qwen3_5, qwen3_5_moe

jamesbraza · 2026-05-04T17:56:36Z

Hi, _BaseAutoModelClass.from_config and .from_pretrained now copy quantization_config onto the unwrapped text config whenever the parent has the attribute, even if the value is None; downstream quantizer detection treats this as pre-quantized and will crash on None.get(...).

Severity: action required | Category: correctness

How to fix: Guard quantization_config propagation

...

Sure @Qodo-Free-For-OSS , just pushed a commit adding this to both call sites in auto_factory.py (from_config and from_pretrained):

# Check both `quantization_config` being present and also not null,
# as a `config.json` can have `"quantization_config": null` in it
parent_quant = getattr(parent_config, "quantization_config", None)
if parent_quant is not None:
    config.quantization_config = parent_quant

zucchini-nlp

Overall I don't mind it, since we already support loading only LM part of Qwen3-5 from_pretrained

My comments aren't really directed to this PR. The fact that we allowed and support loading one model type inside a different one wasn't a good idea imo. As said, slipped off my radar so let's get it in for consistency in from_config/from_pretrained

HuggingFaceDocBuilderDev · 2026-05-05T10:51:41Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

jamesbraza commented May 4, 2026

View reviewed changes

zucchini-nlp reviewed May 4, 2026

View reviewed changes

Comment thread tests/models/gemma3/test_modeling_gemma3.py

zucchini-nlp reviewed May 4, 2026

View reviewed changes

jamesbraza and others added 2 commits May 4, 2026 09:48

jamesbraza force-pushed the fix-automodel-from-config-text-config-unwrap-45759 branch from 80fd48f to 42d9f7a Compare May 4, 2026 16:48

jamesbraza requested a review from zucchini-nlp May 4, 2026 19:01

zucchini-nlp approved these changes May 5, 2026

View reviewed changes

zucchini-nlp added this pull request to the merge queue May 5, 2026

Merged via the queue into huggingface:main with commit 136befe May 5, 2026
29 checks passed

jamesbraza deleted the fix-automodel-from-config-text-config-unwrap-45759 branch May 5, 2026 15:17

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unwrap `text_config` in `AutoModelFor*.from_config`#45770

Unwrap `text_config` in `AutoModelFor*.from_config`#45770
zucchini-nlp merged 3 commits intohuggingface:mainfrom
jamesbraza:fix-automodel-from-config-text-config-unwrap-45759

jamesbraza commented May 4, 2026

Uh oh!

jamesbraza May 4, 2026

Uh oh!

Uh oh!

zucchini-nlp May 4, 2026

Uh oh!

jamesbraza May 4, 2026

Uh oh!

Qodo-Free-For-OSS commented May 4, 2026

Issue description

Issue Context

Fix Focus Areas

What to change

Uh oh!

github-actions Bot commented May 4, 2026

Uh oh!

jamesbraza commented May 4, 2026

Uh oh!

zucchini-nlp left a comment

Uh oh!

HuggingFaceDocBuilderDev commented May 5, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

		# TODO: Validate that copying the parent quantization config to the text sub-config preserves
		# modules_to_not_convert and skip-module matching when composite-model module prefixes differ.

Conversation

jamesbraza commented May 4, 2026

Uh oh!

jamesbraza May 4, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

zucchini-nlp May 4, 2026

Choose a reason for hiding this comment

Uh oh!

jamesbraza May 4, 2026

Choose a reason for hiding this comment

Uh oh!

Qodo-Free-For-OSS commented May 4, 2026

Issue description

Issue Context

Fix Focus Areas

What to change

Uh oh!

github-actions Bot commented May 4, 2026

Uh oh!

jamesbraza commented May 4, 2026

Uh oh!

zucchini-nlp left a comment

Choose a reason for hiding this comment

Uh oh!

HuggingFaceDocBuilderDev commented May 5, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants