How to load Qwen-Image-Lightning's acceleration lora in qwen-image-eidt-2509?

[2502128021]

Hi,
Do you know how to load Qwen-Image-Lightning's acceleration lora in qwen-image-eidt-2509? In DiffSynth-Studio I loaded the acceleration lora and it shows 720 tensors are uploaded by LoRA, but the results were not right. if I set alpha=8, the results are all black, if I set alpha=1, the results are covered by heavy grids.
I use lora in "https://huggingface.co/lightx2v/Qwen-Image-Lightning/tree/main/Qwen-Image-Edit-2509", I've tried Qwen-Image-Edit-2509-Lightning-8steps-V1.0-bf16.safetensors、Qwen-Image-Edit-2509-Lightning-8steps-V1.0-fp32.safetensors, all wrong.
Do you know how to use it correctly? thx!

[yinguoweiOvO]

You can try to convert the lightning lora weight file using the following script, and then use the load_lora function to load the converted weights and set alpha to 1.

from safetensors import safe_open
from safetensors.torch import save_file

input_path = "Qwen-Image-Lightning/Qwen-Image-Edit-2509/Qwen-Image-Edit-2509-Lightning-8steps-V1.0-bf16.safetensors"
output_path = "test_convert.safetensors"
RANK = 64

state_dict = {}
with safe_open(input_path, framework="pt", device="cpu") as f:
    for key in f.keys():
        state_dict[key] = f.get_tensor(key)

new_state_dict = {}
processed_alpha_keys = set()

all_keys = set(state_dict.keys())

for key in sorted(all_keys):
    tensor = state_dict[key]
    if key.endswith("lora_down.weight"):
        prefix = key[:-len("lora_down.weight")] 
        alpha_key = prefix + "alpha"
        if alpha_key in state_dict:
            alpha = state_dict[alpha_key]
            if alpha.numel() == 1:
                scale = alpha.item() / RANK
            else:
                exit()
            scaled_weight = scale * tensor
            processed_alpha_keys.add(alpha_key)
        else:
            scaled_weight = tensor
            print(f"⚠️ Warning: No alpha found for {key}, using scale=1")

        new_key = key.replace("lora_down.weight", "lora_A.default.weight")
        new_state_dict[new_key] = scaled_weight

    elif key.endswith("lora_up.weight"):
        new_key = key.replace("lora_up.weight", "lora_B.default.weight")
        new_state_dict[new_key] = tensor

    else:
        new_state_dict[key] = tensor

save_file(new_state_dict, output_path)

print(f"LoRA 权重已缩放（alpha / {RANK}）并重命名，保存至: {output_path}")
print(f"共保留 {len(new_state_dict)} 个张量，移除了 {len(processed_alpha_keys)} 个 alpha 参数")

[Deng-Xian-Sheng]

use https://github.com/ModelTC/Qwen-Image-Lightning/blob/main/generate_with_diffusers.py

[stillbetter]

You can try to convert the lightning lora weight file using the following script, and then use the load_lora function to load the converted weights and set alpha to 1.

from safetensors import safe_open
from safetensors.torch import save_file

input_path = "Qwen-Image-Lightning/Qwen-Image-Edit-2509/Qwen-Image-Edit-2509-Lightning-8steps-V1.0-bf16.safetensors"
output_path = "test_convert.safetensors"
RANK = 64

state_dict = {}
with safe_open(input_path, framework="pt", device="cpu") as f:
    for key in f.keys():
        state_dict[key] = f.get_tensor(key)

new_state_dict = {}
processed_alpha_keys = set()

all_keys = set(state_dict.keys())

for key in sorted(all_keys):
    tensor = state_dict[key]
    if key.endswith("lora_down.weight"):
        prefix = key[:-len("lora_down.weight")] 
        alpha_key = prefix + "alpha"
        if alpha_key in state_dict:
            alpha = state_dict[alpha_key]
            if alpha.numel() == 1:
                scale = alpha.item() / RANK
            else:
                exit()
            scaled_weight = scale * tensor
            processed_alpha_keys.add(alpha_key)
        else:
            scaled_weight = tensor
            print(f"⚠️ Warning: No alpha found for {key}, using scale=1")

        new_key = key.replace("lora_down.weight", "lora_A.default.weight")
        new_state_dict[new_key] = scaled_weight

    elif key.endswith("lora_up.weight"):
        new_key = key.replace("lora_up.weight", "lora_B.default.weight")
        new_state_dict[new_key] = tensor

    else:
        new_state_dict[key] = tensor

save_file(new_state_dict, output_path)

print(f"LoRA 权重已缩放（alpha / {RANK}）并重命名，保存至: {output_path}")
print(f"共保留 {len(new_state_dict)} 个张量，移除了 {len(processed_alpha_keys)} 个 alpha 参数")

你好，感谢分享代码，请问这样转换是因为modelscope和diffuser对于lora模型的保存方式不一样么？

[stillbetter]

use https://github.com/ModelTC/Qwen-Image-Lightning/blob/main/generate_with_diffusers.py

请问如果想接着Qwen-Image-Edit-2509-Lightning-8steps-V1.0-bf16.safetensors这个lora 继续训练应该怎么做呢？diffusers目前还不支持qwen iamge edit的lora

[yinguoweiOvO]

You can try to convert the lightning lora weight file using the following script, and then use the load_lora function to load the converted weights and set alpha to 1.

from safetensors import safe_open
from safetensors.torch import save_file

input_path = "Qwen-Image-Lightning/Qwen-Image-Edit-2509/Qwen-Image-Edit-2509-Lightning-8steps-V1.0-bf16.safetensors"
output_path = "test_convert.safetensors"
RANK = 64

state_dict = {}
with safe_open(input_path, framework="pt", device="cpu") as f:
    for key in f.keys():
        state_dict[key] = f.get_tensor(key)

new_state_dict = {}
processed_alpha_keys = set()

all_keys = set(state_dict.keys())

for key in sorted(all_keys):
    tensor = state_dict[key]
    if key.endswith("lora_down.weight"):
        prefix = key[:-len("lora_down.weight")] 
        alpha_key = prefix + "alpha"
        if alpha_key in state_dict:
            alpha = state_dict[alpha_key]
            if alpha.numel() == 1:
                scale = alpha.item() / RANK
            else:
                exit()
            scaled_weight = scale * tensor
            processed_alpha_keys.add(alpha_key)
        else:
            scaled_weight = tensor
            print(f"⚠️ Warning: No alpha found for {key}, using scale=1")

        new_key = key.replace("lora_down.weight", "lora_A.default.weight")
        new_state_dict[new_key] = scaled_weight

    elif key.endswith("lora_up.weight"):
        new_key = key.replace("lora_up.weight", "lora_B.default.weight")
        new_state_dict[new_key] = tensor

    else:
        new_state_dict[key] = tensor

save_file(new_state_dict, output_path)

print(f"LoRA 权重已缩放（alpha / {RANK}）并重命名，保存至: {output_path}")
print(f"共保留 {len(new_state_dict)} 个张量，移除了 {len(processed_alpha_keys)} 个 alpha 参数")

你好，感谢分享代码，请问这样转换是因为modelscope和diffuser对于lora模型的保存方式不一样么？

这样转换是因为Diffsynth-Studio中模型的key和lightning发布的加速lora key不对应，不进行转换会导致lora权重加载不上。
另外Qwen-Image-Edit-2509-Lightning-8steps-V1.0-bf16.safetensors是一个蒸馏加速lora，想要接着它训练的话常规训练方式应该是行不通的，会导致加速能力消失。可以直接在Qwen-Image-Edit-2509的基础上训练lora，使用时同时加载自己训练的lora和加速lora就可以了。我还没有成功在此加速lora基础上继续训练过，也许你可以参考此仓库中https://github.com/modelscope/DiffSynth-Studio/blob/main/examples/z_image/model_training/special/trajectory_imitation/Z-Image-Turbo.sh这种训练方式

[stillbetter]

You can try to convert the lightning lora weight file using the following script, and then use the load_lora function to load the converted weights and set alpha to 1.

from safetensors import safe_open
from safetensors.torch import save_file

input_path = "Qwen-Image-Lightning/Qwen-Image-Edit-2509/Qwen-Image-Edit-2509-Lightning-8steps-V1.0-bf16.safetensors"
output_path = "test_convert.safetensors"
RANK = 64

state_dict = {}
with safe_open(input_path, framework="pt", device="cpu") as f:
    for key in f.keys():
        state_dict[key] = f.get_tensor(key)

new_state_dict = {}
processed_alpha_keys = set()

all_keys = set(state_dict.keys())

for key in sorted(all_keys):
    tensor = state_dict[key]
    if key.endswith("lora_down.weight"):
        prefix = key[:-len("lora_down.weight")] 
        alpha_key = prefix + "alpha"
        if alpha_key in state_dict:
            alpha = state_dict[alpha_key]
            if alpha.numel() == 1:
                scale = alpha.item() / RANK
            else:
                exit()
            scaled_weight = scale * tensor
            processed_alpha_keys.add(alpha_key)
        else:
            scaled_weight = tensor
            print(f"⚠️ Warning: No alpha found for {key}, using scale=1")

        new_key = key.replace("lora_down.weight", "lora_A.default.weight")
        new_state_dict[new_key] = scaled_weight

    elif key.endswith("lora_up.weight"):
        new_key = key.replace("lora_up.weight", "lora_B.default.weight")
        new_state_dict[new_key] = tensor

    else:
        new_state_dict[key] = tensor

save_file(new_state_dict, output_path)

print(f"LoRA 权重已缩放（alpha / {RANK}）并重命名，保存至: {output_path}")
print(f"共保留 {len(new_state_dict)} 个张量，移除了 {len(processed_alpha_keys)} 个 alpha 参数")

你好，感谢分享代码，请问这样转换是因为modelscope和diffuser对于lora模型的保存方式不一样么？

这样转换是因为Diffsynth-Studio中模型的key和lightning发布的加速lora key不对应，不进行转换会导致lora权重加载不上。 另外Qwen-Image-Edit-2509-Lightning-8steps-V1.0-bf16.safetensors是一个蒸馏加速lora，想要接着它训练的话常规训练方式应该是行不通的，会导致加速能力消失。可以直接在Qwen-Image-Edit-2509的基础上训练lora，使用时同时加载自己训练的lora和加速lora就可以了。我还没有成功在此加速lora基础上继续训练过，也许你可以参考此仓库中https://github.com/modelscope/DiffSynth-Studio/blob/main/examples/z_image/model_training/special/trajectory_imitation/Z-Image-Turbo.sh这种训练方式

感谢！想再请教下加载这个lora后训练会导致加速能力消失呢？我看Qwen-Image-Lightning的推理脚本，其实也是加载了原生模型然后挂了lora，只不过是修改了scheduler。那如果基于Diffsynth-Studio，是不是也再调整下scheduler即可呢？

[yinguoweiOvO]

You can try to convert the lightning lora weight file using the following script, and then use the load_lora function to load the converted weights and set alpha to 1.

from safetensors import safe_open
from safetensors.torch import save_file

input_path = "Qwen-Image-Lightning/Qwen-Image-Edit-2509/Qwen-Image-Edit-2509-Lightning-8steps-V1.0-bf16.safetensors"
output_path = "test_convert.safetensors"
RANK = 64

state_dict = {}
with safe_open(input_path, framework="pt", device="cpu") as f:
    for key in f.keys():
        state_dict[key] = f.get_tensor(key)

new_state_dict = {}
processed_alpha_keys = set()

all_keys = set(state_dict.keys())

for key in sorted(all_keys):
    tensor = state_dict[key]
    if key.endswith("lora_down.weight"):
        prefix = key[:-len("lora_down.weight")] 
        alpha_key = prefix + "alpha"
        if alpha_key in state_dict:
            alpha = state_dict[alpha_key]
            if alpha.numel() == 1:
                scale = alpha.item() / RANK
            else:
                exit()
            scaled_weight = scale * tensor
            processed_alpha_keys.add(alpha_key)
        else:
            scaled_weight = tensor
            print(f"⚠️ Warning: No alpha found for {key}, using scale=1")

        new_key = key.replace("lora_down.weight", "lora_A.default.weight")
        new_state_dict[new_key] = scaled_weight

    elif key.endswith("lora_up.weight"):
        new_key = key.replace("lora_up.weight", "lora_B.default.weight")
        new_state_dict[new_key] = tensor

    else:
        new_state_dict[key] = tensor

save_file(new_state_dict, output_path)

print(f"LoRA 权重已缩放（alpha / {RANK}）并重命名，保存至: {output_path}")
print(f"共保留 {len(new_state_dict)} 个张量，移除了 {len(processed_alpha_keys)} 个 alpha 参数")

你好，感谢分享代码，请问这样转换是因为modelscope和diffuser对于lora模型的保存方式不一样么？

这样转换是因为Diffsynth-Studio中模型的key和lightning发布的加速lora key不对应，不进行转换会导致lora权重加载不上。 另外Qwen-Image-Edit-2509-Lightning-8steps-V1.0-bf16.safetensors是一个蒸馏加速lora，想要接着它训练的话常规训练方式应该是行不通的，会导致加速能力消失。可以直接在Qwen-Image-Edit-2509的基础上训练lora，使用时同时加载自己训练的lora和加速lora就可以了。我还没有成功在此加速lora基础上继续训练过，也许你可以参考此仓库中https://github.com/modelscope/DiffSynth-Studio/blob/main/examples/z_image/model_training/special/trajectory_imitation/Z-Image-Turbo.sh这种训练方式

感谢！想再请教下加载这个lora后训练会导致加速能力消失呢？我看Qwen-Image-Lightning的推理脚本，其实也是加载了原生模型然后挂了lora，只不过是修改了scheduler。那如果基于Diffsynth-Studio，是不是也再调整下scheduler即可呢？

您说的是推理脚本，而不是训练脚本。lightning的加速能力是通过特殊的训练方式赋予的，而加载此lora再以常规方式训练自己的lora会破坏这种加速能力

[stillbetter]

You can try to convert the lightning lora weight file using the following script, and then use the load_lora function to load the converted weights and set alpha to 1.

from safetensors import safe_open
from safetensors.torch import save_file

input_path = "Qwen-Image-Lightning/Qwen-Image-Edit-2509/Qwen-Image-Edit-2509-Lightning-8steps-V1.0-bf16.safetensors"
output_path = "test_convert.safetensors"
RANK = 64

state_dict = {}
with safe_open(input_path, framework="pt", device="cpu") as f:
    for key in f.keys():
        state_dict[key] = f.get_tensor(key)

new_state_dict = {}
processed_alpha_keys = set()

all_keys = set(state_dict.keys())

for key in sorted(all_keys):
    tensor = state_dict[key]
    if key.endswith("lora_down.weight"):
        prefix = key[:-len("lora_down.weight")] 
        alpha_key = prefix + "alpha"
        if alpha_key in state_dict:
            alpha = state_dict[alpha_key]
            if alpha.numel() == 1:
                scale = alpha.item() / RANK
            else:
                exit()
            scaled_weight = scale * tensor
            processed_alpha_keys.add(alpha_key)
        else:
            scaled_weight = tensor
            print(f"⚠️ Warning: No alpha found for {key}, using scale=1")

        new_key = key.replace("lora_down.weight", "lora_A.default.weight")
        new_state_dict[new_key] = scaled_weight

    elif key.endswith("lora_up.weight"):
        new_key = key.replace("lora_up.weight", "lora_B.default.weight")
        new_state_dict[new_key] = tensor

    else:
        new_state_dict[key] = tensor

save_file(new_state_dict, output_path)

print(f"LoRA 权重已缩放（alpha / {RANK}）并重命名，保存至: {output_path}")
print(f"共保留 {len(new_state_dict)} 个张量，移除了 {len(processed_alpha_keys)} 个 alpha 参数")

你好，感谢分享代码，请问这样转换是因为modelscope和diffuser对于lora模型的保存方式不一样么？

这样转换是因为Diffsynth-Studio中模型的key和lightning发布的加速lora key不对应，不进行转换会导致lora权重加载不上。 另外Qwen-Image-Edit-2509-Lightning-8steps-V1.0-bf16.safetensors是一个蒸馏加速lora，想要接着它训练的话常规训练方式应该是行不通的，会导致加速能力消失。可以直接在Qwen-Image-Edit-2509的基础上训练lora，使用时同时加载自己训练的lora和加速lora就可以了。我还没有成功在此加速lora基础上继续训练过，也许你可以参考此仓库中https://github.com/modelscope/DiffSynth-Studio/blob/main/examples/z_image/model_training/special/trajectory_imitation/Z-Image-Turbo.sh这种训练方式

感谢！想再请教下加载这个lora后训练会导致加速能力消失呢？我看Qwen-Image-Lightning的推理脚本，其实也是加载了原生模型然后挂了lora，只不过是修改了scheduler。那如果基于Diffsynth-Studio，是不是也再调整下scheduler即可呢？

您说的是推理脚本，而不是训练脚本。lightning的加速能力是通过特殊的训练方式赋予的，而加载此lora再以常规方式训练自己的lora会破坏这种加速能力

感谢，我上文想表达正是推理脚本额外多了一个schedule的调整，所以我想这一部分可能也是加速的关键？lightning lora应该与匹配的schedule一起使用。

[yinguoweiOvO]

You can try to convert the lightning lora weight file using the following script, and then use the load_lora function to load the converted weights and set alpha to 1.

from safetensors import safe_open
from safetensors.torch import save_file

input_path = "Qwen-Image-Lightning/Qwen-Image-Edit-2509/Qwen-Image-Edit-2509-Lightning-8steps-V1.0-bf16.safetensors"
output_path = "test_convert.safetensors"
RANK = 64

state_dict = {}
with safe_open(input_path, framework="pt", device="cpu") as f:
    for key in f.keys():
        state_dict[key] = f.get_tensor(key)

new_state_dict = {}
processed_alpha_keys = set()

all_keys = set(state_dict.keys())

for key in sorted(all_keys):
    tensor = state_dict[key]
    if key.endswith("lora_down.weight"):
        prefix = key[:-len("lora_down.weight")] 
        alpha_key = prefix + "alpha"
        if alpha_key in state_dict:
            alpha = state_dict[alpha_key]
            if alpha.numel() == 1:
                scale = alpha.item() / RANK
            else:
                exit()
            scaled_weight = scale * tensor
            processed_alpha_keys.add(alpha_key)
        else:
            scaled_weight = tensor
            print(f"⚠️ Warning: No alpha found for {key}, using scale=1")

        new_key = key.replace("lora_down.weight", "lora_A.default.weight")
        new_state_dict[new_key] = scaled_weight

    elif key.endswith("lora_up.weight"):
        new_key = key.replace("lora_up.weight", "lora_B.default.weight")
        new_state_dict[new_key] = tensor

    else:
        new_state_dict[key] = tensor

save_file(new_state_dict, output_path)

print(f"LoRA 权重已缩放（alpha / {RANK}）并重命名，保存至: {output_path}")
print(f"共保留 {len(new_state_dict)} 个张量，移除了 {len(processed_alpha_keys)} 个 alpha 参数")

你好，感谢分享代码，请问这样转换是因为modelscope和diffuser对于lora模型的保存方式不一样么？

这样转换是因为Diffsynth-Studio中模型的key和lightning发布的加速lora key不对应，不进行转换会导致lora权重加载不上。 另外Qwen-Image-Edit-2509-Lightning-8steps-V1.0-bf16.safetensors是一个蒸馏加速lora，想要接着它训练的话常规训练方式应该是行不通的，会导致加速能力消失。可以直接在Qwen-Image-Edit-2509的基础上训练lora，使用时同时加载自己训练的lora和加速lora就可以了。我还没有成功在此加速lora基础上继续训练过，也许你可以参考此仓库中https://github.com/modelscope/DiffSynth-Studio/blob/main/examples/z_image/model_training/special/trajectory_imitation/Z-Image-Turbo.sh这种训练方式

感谢！想再请教下加载这个lora后训练会导致加速能力消失呢？我看Qwen-Image-Lightning的推理脚本，其实也是加载了原生模型然后挂了lora，只不过是修改了scheduler。那如果基于Diffsynth-Studio，是不是也再调整下scheduler即可呢？

您说的是推理脚本，而不是训练脚本。lightning的加速能力是通过特殊的训练方式赋予的，而加载此lora再以常规方式训练自己的lora会破坏这种加速能力

感谢，我上文想表达正是推理脚本额外多了一个schedule的调整，所以我想这一部分可能也是加速的关键？lightning lora应该与匹配的schedule一起使用。

这块我没有仔细看过，不过貌似使用此仓库的scheduler设置cfg=1，step=8能够正常出图～