Civit AI flux model razor-8step-rapid-real not working with diffusers single file

[ali-afridi26]

Describe the bug

We have this civit AI model: https://civitai.com/models/849864/razor-8step-rapid-real which we want to run using from_single_file, but it errors out

Reproduction

1. First create your CivitAI API key by logging into civit ai and navigating to https://civitai.com/user/account
   Then go to "API Keys" section in the bottom and create your key.
2. Run the following command on terminal: wget --show-progress -O model.safetensors "https://api.civitai.com/download/models/950841?token=YOUR_TOKEN"
3. Try the code:

import torch
from diffusers import FluxPipeline

#wget --show-progress -O model.safetensors "https://api.civitai.com/download/models/950841?token="

pipe = FluxPipeline.from_single_file(
    "model.safetensors",
    torch_dtype=torch.bfloat16,
    
)
pipe.to("cuda")
prompt = "A cat holding a sign that says hello world"
image = pipe(
    prompt,
    height=1024,
    width=1024,
    guidance_scale=3.5,
    num_inference_steps=50,
    max_sequence_length=512,
    generator=torch.Generator("cuda").manual_seed(0)
).images[0]
image.save("flux.png")

Logs

(3.7) user@c6dbd33b-904f-4d4e-bc4e-f68f78a80315:~/runware/Ali/sd-base-api$ python ali.py 
Fetching 16 files: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████| 16/16 [00:00<00:00, 24745.16it/s]
Loading pipeline components...:  57%|█████████████████████████████████████████████████████████▏                                          | 4/7 [00:16<00:12,  4.16s/it]You set `add_prefix_space`. The tokenizer needs to be converted from the slow tokenizers
Loading pipeline components...:  71%|███████████████████████████████████████████████████████████████████████▍                            | 5/7 [00:16<00:06,  3.37s/it]
Traceback (most recent call last):
  File "/home/user/runware/Ali/sd-base-api/ali.py", line 7, in <module>
    pipe = FluxPipeline.from_single_file(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/anaconda3/envs/3.7/lib/python3.12/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/user/anaconda3/envs/3.7/lib/python3.12/site-packages/diffusers/loaders/single_file.py", line 509, in from_single_file
    loaded_sub_model = load_single_file_sub_model(
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/anaconda3/envs/3.7/lib/python3.12/site-packages/diffusers/loaders/single_file.py", line 104, in load_single_file_sub_model
    loaded_sub_model = load_method(
                       ^^^^^^^^^^^^
  File "/home/user/anaconda3/envs/3.7/lib/python3.12/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/home/user/anaconda3/envs/3.7/lib/python3.12/site-packages/diffusers/loaders/single_file_model.py", line 343, in from_single_file
    diffusers_format_checkpoint = checkpoint_mapping_fn(
                                  ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/anaconda3/envs/3.7/lib/python3.12/site-packages/diffusers/loaders/single_file_utils.py", line 2255, in convert_flux_transformer_checkpoint_to_diffusers
    q, k, v, mlp = torch.split(checkpoint.pop(f"single_blocks.{i}.linear1.weight"), split_size, dim=0)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/anaconda3/envs/3.7/lib/python3.12/site-packages/torch/functional.py", line 207, in split
    return tensor.split(split_size_or_sections, dim)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/anaconda3/envs/3.7/lib/python3.12/site-packages/torch/_tensor.py", line 983, in split
    return torch._VF.split_with_sizes(self, split_size, dim)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: split_with_sizes expects split_sizes to sum exactly to 33030144 (input tensor's size at dimension 0), but got split_sizes=[3072, 3072, 3072, 12288]

System Info

Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

• 🤗 Diffusers version: 0.33.0.dev0
• Platform: Linux-5.15.0-134-generic-x86_64-with-glibc2.35
• Running on Google Colab?: No
• Python version: 3.12.9
• PyTorch version (GPU?): 2.5.1+cu124 (True)
• Flax version (CPU?/GPU?/TPU?): not installed (NA)
• Jax version: not installed
• JaxLib version: not installed
• Huggingface_hub version: 0.27.1
• Transformers version: 4.47.1
• Accelerate version: 1.2.1
• PEFT version: 0.14.0
• Bitsandbytes version: not installed
• Safetensors version: 0.5.0
• xFormers version: not installed
• Accelerator: NVIDIA A100-SXM4-80GB, 81920 MiB
• Using GPU in script?: Yes
• Using distributed or parallel set-up in script?: No

Who can help?

@sayakpaul

[hlky]

Hi @ali-afridi26. I've edited to remove your civit api key and deleted the revisions.

[ali-afridi26]

Thanks, appreciated!

[CyberVy]

I think maybe the file your downloaded from civitai is only a FLUX transformer model.

This means you can't load it by FluxPipeline.from_single_file, which requires all FLUX components. You can load the transformer, text encoders, tokenizers, vae and scheduler first, and then pass them to FluxPipeline.__init__.

[nitinmukesh]

import torch
from diffusers import FluxPipeline, FluxTransformer2DModel

model_id ="black-forest-labs/FLUX.1-dev"
dtype = torch.bfloat16
transformer_path = "model.safetensors"
transformer = FluxTransformer2DModel.from_single_file(
	transformer_path,
	torch_dtype=dtype,
)

pipe = FluxPipeline.from_pretrained(
	model_id,
	transformer=transformer,
	torch_dtype=dtype,
)
pipe.to("cuda")
prompt = "A cat holding a sign that says hello world"
image = pipe(
    prompt,
    height=1024,
    width=1024,
    guidance_scale=3.5,
    num_inference_steps=50,
    max_sequence_length=512,
    generator=torch.Generator("cuda").manual_seed(0)
).images[0]
image.save("flux.png")

@ali-afridi26
You are loading transformer only. See if this code works

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[yiyixuxu]

is this issue resolved?

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.