Cannot load models with bf16 norm ".scale" weights

[stduhpf]

https://huggingface.co/lodestones/Chroma1-Radiance/discussions/12

If the type of the norm scale tensor is b16, the model won't load, with this error:

VAE load device: cuda:0, offload device: cpu, dtype: torch.bfloat16
Requested to load PixelspaceConversionVAE
loaded completely; 14486.01 MB usable, 0.00 MB loaded, full load: True
Warning: This gguf model file is loaded in compatibility mode 'sd.cpp' [arch:flux]
gguf qtypes: BF16 (424), F32 (3), Q4_K (190), Q5_K (38), Q8_0 (4)
model weight dtype torch.bfloat16, manual cast: None
model_type FLUX
!!! Exception during processing !!! Error(s) in loading state_dict for ChromaRadiance:
        While copying the parameter named "distilled_guidance_layer.norms.0.scale", whose dimensions in the model are torch.Size([5120]) and whose dimensions in the checkpoint are torch.Size([10240]), an exception occurred : ('The size of tensor a (5120) must match the size of tensor b (10240) at non-singleton dimension 0',).
        While copying the parameter named "distilled_guidance_layer.norms.1.scale", whose dimensions in the model are torch.Size([5120]) and whose dimensions in the checkpoint are torch.Size([10240]), an exception occurred : ('The size of tensor a (5120) must match the size of tensor b (10240) at non-singleton dimension 0',).
        While copying the parameter named "distilled_guidance_layer.norms.2.scale", whose dimensions in the model are torch.Size([5120]) and whose dimensions in the checkpoint are torch.Size([10240]), an exception occurred : ('The size of tensor a (5120) must match the size of tensor b (10240) at non-singleton dimension 0',).
        While copying the parameter named "distilled_guidance_layer.norms.3.scale", whose dimensions in the model are torch.Size([5120]) and whose dimensions in the checkpoint are torch.Size([10240]), an exception occurred : ('The size of tensor a (5120) must match the size of tensor b (10240) at non-singleton dimension 0',).
        While copying the parameter named "distilled_guidance_layer.norms.4.scale", whose dimensions in the model are torch.Size([5120]) and whose dimensions in the checkpoint are torch.Size([10240]), an exception occurred : ('The size of tensor a (5120) must match the size of tensor b (10240) at non-singleton dimension 0',).
        While copying the parameter named "double_blocks.0.img_attn.norm.query_norm.scale", whose dimensions in the model are torch.Size([128]) and whose dimensions in the checkpoint are torch.Size([256]), an exception occurred : ('The size of tensor a (128) must match the size of tensor b (256) at non-singleton dimension 0',).
         [ and a lot more for every ".scale" tensor in the model]
Traceback (most recent call last):
  File "H:\ComfyUI\execution.py", line 516, in execute
    output_data, output_ui, has_subgraph, has_pending_tasks = await get_output_data(prompt_id, unique_id, obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
  File "H:\ComfyUI\execution.py", line 330, in get_output_data
    return_values = await _async_map_node_over_list(prompt_id, unique_id, obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb, v3_data=v3_data)
  File "H:\ComfyUI\execution.py", line 304, in _async_map_node_over_list
    await process_inputs(input_dict, i)
  File "H:\ComfyUI\execution.py", line 292, in process_inputs
    result = f(**inputs)
  File "H:\ComfyUI\custom_nodes\ComfyUI-GGUF\nodes.py", line 153, in load_unet
    model = comfy.sd.load_diffusion_model_state_dict(
  File "H:\ComfyUI\comfy\sd.py", line 1496, in load_diffusion_model_state_dict
    model.load_model_weights(new_sd, "")
  File "H:\ComfyUI\comfy\model_base.py", line 308, in load_model_weights
    m, u = self.diffusion_model.load_state_dict(to_load, strict=False)
  File "H:\ComfyUI\venv\lib\site-packages\torch\nn\modules\module.py", line 2189, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format([same spam as above]):

It seems to detect each bf16 weights as if it was two elements?

[city96]

Pushed a fix for that, though the conversion code in this repo shouldn't actually allow 1D tensors to be quantized (they're always stored in FP32). Still, now we just dequantize them on the fly for external models that do have them like the one you linked.