I can run Kolors on a quant4 setup, but loading the chatglm3-4bit.safetensors model separately gives an error!

[klossm]

Error when loading chatglm3-4bit.safetensors

Traceback (most recent call last):
File "L:\stable-diffusion\extensions\sd-webui-comfyui\ComfyUI\execution.py", line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "L:\stable-diffusion\extensions\sd-webui-comfyui\ComfyUI\execution.py", line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "L:\stable-diffusion\extensions\sd-webui-comfyui\ComfyUI\execution.py", line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "L:\stable-diffusion\extensions\sd-webui-comfyui\ComfyUI\custom_nodes\ComfyUI-KwaiKolorsWrapper\nodes.py", line 122, in loadmodel
text_encoder.quantize(4)
File "L:\stable-diffusion\extensions\sd-webui-comfyui\ComfyUI\custom_nodes\ComfyUI-KwaiKolorsWrapper\kolors\models\modeling_chatglm.py", line 852, in quantize
quantize(self.encoder, weight_bit_width)
File "L:\stable-diffusion\extensions\sd-webui-comfyui\ComfyUI\custom_nodes\ComfyUI-KwaiKolorsWrapper\kolors\models\quantization.py", line 155, in quantize
layer.self_attention.query_key_value = QuantizedLinear(
File "L:\stable-diffusion\extensions\sd-webui-comfyui\ComfyUI\custom_nodes\ComfyUI-KwaiKolorsWrapper\kolors\models\quantization.py", line 141, in init
self.weight = Parameter(self.weight.to(device), requires_grad=False)
File "L:\stable-diffusion\venv\lib\site-packages\torch\nn\modules\module.py", line 1726, in setattr
self.register_parameter(name, value)
File "L:\stable-diffusion\venv\lib\site-packages\accelerate\big_modeling.py", line 105, in register_empty_parameter
module._parameters[name] = param_cls(module._parameters[name].to(device), **kwargs)
File "L:\stable-diffusion\venv\lib\site-packages\torch\nn\parameter.py", line 40, in new
return torch.Tensor._make_subclass(cls, data, requires_grad)
RuntimeError: Only Tensors of floating point and complex dtype can require gradients