`ignore_layers` does not take effect on FP8 model

[yiliu30]

Description:
When running quantization on an FP8 model with the --ignore_layers argument, the specified layers are not ignored as expected. The log shows not_to_quantized_layers: [], indicating the argument is not being processed or respected for FP8 models.

AR_LOG_LEVEL=TRACE auto_round --model /models/Qwen3-8B-FP8 --ignore_layers "mlp"

Loading checkpoint shards: 100%|████████████████████████| 2/2 [00:00<00:00, 16.29it/s]
2026-01-14 22:03:32 WARNING model.py L323: the support for fp8 model as input is experimental, please use with caution.
2026-01-14 22:03:32 INFO base.py L442: using torch.bfloat16 for quantization tuning
2026-01-14 22:03:32 INFO base.py L708: 'enable_torch_compile' is set to `False` by default. Enabling it can reduce tuning cost by 20%, but it might throw an exception.
2026-01-14 22:03:32 DEBUG replace_modules.py L152: Scanning for modules to replace
2026-01-14 22:03:32 DEBUG replace_modules.py L174: No modules found for replacement
2026-01-14 22:03:32 TRACE utils.py L889: not_to_quantized_layers: [] <--------------------

[yiliu30]

I temporarily fix it by adding FP8Linear into SUPPORTED_LAYER_TYPES.
@n1ck-guo @xin3he could you please find a robust solution? Thanks!

auto-round/auto_round/compressors/utils.py

Lines 867 to 876 in 6e1ea9c

	from auto_round.utils import SUPPORTED_LAYER_TYPES
	if not ignore_layers:
	return []
	ignore_layers = ignore_layers.replace(" ", "").split(",")
	all_layer_names = []
	for n, m in model.named_modules():
	if type(m) in SUPPORTED_LAYER_TYPES:
	all_layer_names.append(n)
	not_to_quantized_layers = []

[scopophobic]

I’ve opened a PR that fixes FP8 layer detection so that ignore_layers is correctly populated for FP8 models: #1289

This addresses the root cause behind ignore_layers being ignored in FP8 cases.
Note that FP8 → BF16 dequantization is not yet handled in this PR; that will be addressed in a follow-up (see #1284).

[yiliu30]

This PR fixed part of this issue. #1286
Since INNER_SUPPORTED_LAYER_TYPES only includes FP8Linear, but we actually support other FP8 layer types as well, let’s keep this issue open. Synced with @n1ck-guo offline, we may need refactor that part more robust.

auto-round/auto_round/utils/common.py

Line 165 in ba48dd7

INNER_SUPPORTED_LAYER_TYPES = ("FP8Linear",)

auto-round/auto_round/utils/model.py

Lines 1014 to 1018 in ba48dd7

	if layer.__class__.__name__ == "CompressedLinear":
	dq_weight = layer.compressor.decompress_module(layer)
	else:
	weight_scale = layer.weight_scale if hasattr(layer, "weight_scale") else layer.weight_scale_inv
	dq_weight = dequant_block_fp8_weight(layer.weight, weight_scale, layer.block_size)

[yiliu30]

Hi @yiliu30, thanks a lot for the clarification, that helped resolve a misunderstanding I had 👍
I now understand that the goal is not to skip all FP8 layers, but to start from an FP8 model and re-quantize it (e.g., to W4A16), while keeping specific submodules (like attention) from being quantized.
I’m definitely interested in working on the remaining part of #1283. My current thought is to make FP8 detection more robust by moving away from class-name checks (like "FP8Linear") and instead relying on explicit FP8 characteristics (e.g., presence of FP8 scale metadata used during dequantization). This would allow supporting multiple FP8 layer implementations without brittle heuristics.
Does this approach sound aligned with what you had in mind for this issue?

Hi @scopophobic let's discuss the details here.
The method you mentioned makes sense, but for now we still rely on the original FP8 linear modules to provide some additional information—such as scale names and quantization granularity—for dequantization. Because of that, it’s safer to add support for each layer type incrementally. Over time, we can factor out some common primitives to unify the dequant logic.

Currently, we support two types:

auto-round/auto_round/utils/model.py

Lines 1014 to 1018 in ba48dd7

	if layer.__class__.__name__ == "CompressedLinear":
	dq_weight = layer.compressor.decompress_module(layer)
	else:
	weight_scale = layer.weight_scale if hasattr(layer, "weight_scale") else layer.weight_scale_inv
	dq_weight = dequant_block_fp8_weight(layer.weight, weight_scale, layer.block_size)

To extend this more cleanly, we could introduce a registration mechanism or base class, so the supported list stays concise and easier to maintain. For example:

For example:

@register_fp8_layer("CompressedLinear")
def dequant_weight_func(layer: CompressedLinear):
    # return the dequantized weight for this layer

@register_fp8_layer("FP8Linear")
def dequant_weight_func(layer: FP8Linear):
    # return the dequantized weight for this layer

This would give us a scalable way to support new FP8 layer types while keeping the logic well-organized.
I’d love to hear your thoughts—thanks!