[Configs] Fix layer type validation to include its mlp counterpart

[vasqu]

Slight rewrite to make the validation of layer type to include mlp layer types (2 separate lists - 1 is attn (e.g. full vs swa) 1 is mlp (dense vs sparse)). Probably got lost in the refactor for strict validation

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[Cyrilvallez]

Nice thanks! Would be nice to separate mlp and attention in the future though probably!

[Cyrilvallez]

Let's raise more precise error, does not cost much haha

Suggested change

	elif not all(layer_type in ALLOWED_LAYER_TYPES for layer_type in layers):
	raise ValueError(f"The `(mlp)_layer_types` entries must be in {ALLOWED_LAYER_TYPES} but got {layers}")
	elif not all(layer_type in ALLOWED_LAYER_TYPES for layer_type in layers):
	raise ValueError(f"The `{layer_types}` entries must be in {ALLOWED_LAYER_TYPES} but got {layers}")

[vasqu]

Done, also fixed the message below

[zucchini-nlp]

mlp layer types have different set of acceptable names, no? I'd prefer to add a separate self.validate_mlp_layer_types in this PR before merging since it's not a huge change

[vasqu]

It used to be different sets but apparently they got merged into one 🤷‍♂️

[vasqu]

transformers/src/transformers/configuration_utils.py

Lines 62 to 76 in f39b5c8

	ALLOWED_LAYER_TYPES = (
	"full_attention",
	"sliding_attention",
	"chunked_attention",
	"compressed_sparse_attention", # CSA, used in deepseek_v4
	"heavily_compressed_attention", # HCA, used in deepseek_v4
	"linear_attention", # used in minimax
	"conv", # used in LFMv2
	"mamba",
	"attention",
	"sparse",
	"dense",
	"hybrid", # for layers that have both mamba and attention in zamba and zamba2
	"moe", # for nemotron_h, which uses either attention, mamba or moe
	)

[zucchini-nlp]

😢

[vasqu]

Should I still merge or do we want to be breaking here 😓 it's kind of a weird situation

[zucchini-nlp]

fine with merging, I assumed there are already non-overlapping sets for two keys. We can create an issue and fix it later for the sake of "correctness"

[vasqu]

Opened #46245 to keep track, this will be likely breaking