Skip to content

Commit

Permalink
Change the default value for XPU weight-only quantization (#1194)
Browse files Browse the repository at this point in the history
Signed-off-by: Cheng Penghui <penghui.cheng@intel.com>
  • Loading branch information
PenghuiCheng committed Jan 26, 2024
1 parent 7156218 commit 4a78bab
Showing 1 changed file with 1 addition and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -232,7 +232,7 @@ def from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs):
quantization_config = WeightOnlyQuantConfig(compute_dtype="fp32", weight_dtype="nf4")
else:
quantization_config = WeightOnlyQuantConfig(compute_dtype=convert_dtype_torch2str(torch_dtype),
weight_dtype="nf4")
weight_dtype="nf4" if use_cpu else "int4_fullrange")
else:
assert ("4" in quantization_config.weight_dtype
and convert_dtype_str2torch(quantization_config.compute_dtype) == torch_dtype
Expand Down

0 comments on commit 4a78bab

Please sign in to comment.