-
Notifications
You must be signed in to change notification settings - Fork 78
Closed
Labels
bugSomething isn't workingSomething isn't working
Description
Problem Description
This issue does not occur on Qwen/Qwen3‑30B‑A3B; it may be caused by the experts’ linear layer names or FP8 model only?
The get_act_max_hook was missed for dequantized model.
Not sure if it was supported before or not, looks like a feature gap. cc @xin3he @wenhuach21
Reproduction Steps
import torch
from auto_round import AutoRound
model_name = "/storage/yiliu7/MiniMaxAI/MiniMax-M2.1/"
scheme = "FP8_STATIC"
autoround = AutoRound(
model_name,
scheme=scheme,
iters=0,
low_gpu_mem_usage=True,
disable_opt_rtn=True,
)
autoround.quantize_and_save(format="auto_round", output_dir="tmp_autoround_qwen3_fix")Environment Information
model: /storage/yiliu7/MiniMaxAI/MiniMax-M2.1/
commit: 28839a9
Error Logs
- log
model-00002-of-00007.safetensors,model.layers.1.block_sparse_moe.experts.86.w2.input_scale,torch.bfloat16,[1],1.0,1.0
model-00002-of-00007.safetensors,model.layers.1.block_sparse_moe.experts.86.w3.input_scale,torch.bfloat16,[1],1.0,1.0
model-00002-of-00007.safetensors,model.layers.1.block_sparse_moe.experts.87.w1.input_scale,torch.bfloat16,[1],1.0,1.0
model-00002-of-00007.safetensors,model.layers.1.block_sparse_moe.experts.87.w2.input_scale,torch.bfloat16,[1],1.0,1.0
model-00002-of-00007.safetensors,model.layers.1.block_sparse_moe.experts.87.w3.input_scale,torch.bfloat16,[1],1.0,1.0
model-00002-of-00007.safetensors,model.layers.1.block_sparse_moe.experts.88.w1.input_scale,torch.bfloat16,[1],1.0,1.0
model-00002-of-00007.safetensors,model.layers.1.block_sparse_moe.experts.88.w2.input_scale,torch.bfloat16,[1],1.0,1.0
model-00002-of-00007.safetensors,model.layers.1.block_sparse_moe.experts.88.w3.input_scale,torch.bfloat16,[1],1.0,1.0
model-00002-of-00007.safetensors,model.layers.1.block_sparse_moe.experts.89.w1.input_scale,torch.bfloat16,[1],1.0,1.0
model-00002-of-00007.safetensors,model.layers.1.block_sparse_moe.experts.89.w2.input_scale,torch.bfloat16,[1],1.0,1.0
model-00002-of-00007.safetensors,model.layers.1.block_sparse_moe.experts.89.w3.input_scale,torch.bfloat16,[1],1.0,1.0
model-00002-of-00007.safetensors,model.layers.1.block_sparse_moe.experts.9.w1.input_scale,torch.bfloat16,[1],1.0,1.0
...Additional Context
No response
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working