[Bug]: Use auto round quant minimax m2.7, the moe gate weight will be quanted as default.

[a32543254]

Problem Description

Use auto round quant minimax m2.7, the moe gate weight will be quanted as default.
Script

import sys
import argparse
from auto_round import AutoRound

'''
Example:
python convert_model_to_fp8.py --input PATH/to/model --out_file PATH/to/output
'''
parser = argparse.ArgumentParser(description='Convert a hf model to a hf fp8 static quantized model')
parser.add_argument('--input', type=str, help="directory containing model file")
parser.add_argument('--out_file', type=str, help='path to write to')
args = parser.parse_args()

model_path = args.input
save_path = args.out_file


layer_config = {}


ar = AutoRound(
    model=model_path,
    bits=8,
    iters=0,
    act_bits=8,
    layer_config=layer_config,
    group_size=-1,
    enable_torch_compile=True,
    act_group_size=0,
    static_attention_dtype="fp8",
    act_dynamic=False,
    data_type="fp8",
    act_data_type="fp8")
ar.quantize_and_save(save_path, format="auto_round")

the 0-6 layers moe gate will be bf16 dtype
but start from layer 7 the moe gate weight will be quanted to fp8 dtype

Reproduction Steps

python convert_model_to_fp8.py --input PATH/to/model --out_file PATH/to/output
check
MiniMax-M2.7-fp-w8afp8/model.safetensors.index.json

Environment Information

Both cuda and gaudi could reproduce this issue

Error Logs

   "model.layers.10.block_sparse_moe.gate.input_scale": "model-00008-of-00047.safetensors",
    "model.layers.10.block_sparse_moe.gate.weight": "model-00008-of-00047.safetensors",
    "model.layers.10.block_sparse_moe.gate.weight_scale": "model-00008-of-00047.safetensors",

Additional Context

No response

[thuang6]

@xin3he, pls help to take a look