2026-05-19 14:57:34 WARNING llm.py L145: Some layers are offloaded to cpu, which may severely impact calibration speed. Please consider using more cards.
Traceback (most recent call last):
File "/home/sdp/miniforge3/envs/jenkins-key-model-test/bin/auto-round", line 6, in <module>
File "/home/sdp/miniforge3/envs/jenkins-key-model-test/lib/python3.12/site-packages/auto_round/__main__.py", line 834, in run
File "/home/sdp/miniforge3/envs/jenkins-key-model-test/lib/python3.12/site-packages/auto_round/__main__.py", line 570, in start
File "/home/sdp/miniforge3/envs/jenkins-key-model-test/lib/python3.12/site-packages/auto_round/__main__.py", line 775, in tune
File "/home/sdp/miniforge3/envs/jenkins-key-model-test/lib/python3.12/site-packages/auto_round/compressors/base.py", line 1486, in quantize_and_save
File "/home/sdp/miniforge3/envs/jenkins-key-model-test/lib/python3.12/site-packages/auto_round/compressors/data_driven.py", line 654, in quantize
File "/home/sdp/miniforge3/envs/jenkins-key-model-test/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context
File "/home/sdp/miniforge3/envs/jenkins-key-model-test/lib/python3.12/site-packages/auto_round/compressors/data_driven.py", line 146, in try_cache_inter_data_gpucpu
File "/home/sdp/miniforge3/envs/jenkins-key-model-test/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 124, in decorate_context
File "/home/sdp/miniforge3/envs/jenkins-key-model-test/lib/python3.12/site-packages/auto_round/calibration/llm.py", line 151, in collect
File "/home/sdp/miniforge3/envs/jenkins-key-model-test/lib/python3.12/site-packages/accelerate/big_modeling.py", line 432, in dispatch_model
File "/home/sdp/miniforge3/envs/jenkins-key-model-test/lib/python3.12/site-packages/accelerate/hooks.py", line 695, in attach_align_device_hook_on_blocks
File "/home/sdp/miniforge3/envs/jenkins-key-model-test/lib/python3.12/site-packages/accelerate/hooks.py", line 695, in attach_align_device_hook_on_blocks
File "/home/sdp/miniforge3/envs/jenkins-key-model-test/lib/python3.12/site-packages/accelerate/hooks.py", line 695, in attach_align_device_hook_on_blocks
[Previous line repeated 1 more time]
File "/home/sdp/miniforge3/envs/jenkins-key-model-test/lib/python3.12/site-packages/accelerate/hooks.py", line 653, in attach_align_device_hook_on_blocks
File "/home/sdp/miniforge3/envs/jenkins-key-model-test/lib/python3.12/site-packages/accelerate/hooks.py", line 183, in add_hook_to_module
File "/home/sdp/miniforge3/envs/jenkins-key-model-test/lib/python3.12/site-packages/accelerate/hooks.py", line 305, in init_hook
File "/home/sdp/miniforge3/envs/jenkins-key-model-test/lib/python3.12/site-packages/accelerate/utils/modeling.py", line 335, in set_module_tensor_to_device
RuntimeError: level_zero backend failed with error: 2147483646 (UR_RESULT_ERROR_UNKNOWN)
Problem Description
Qwen/Qwen3.6-35B-A3B W4A16 quant failed on BMG
Reproduction Steps
export ZE_AFFINITY_MASK=4,5
auto-round --model Qwen/Qwen3.6-35B-A3B --scheme "W4A16" --format "auto_round" --output_dir ./Qwen3.6-35B-A3B-AutoRound --device_map "auto"
Environment Information
B60
Error Logs
Additional Context
No response