[LoRA] add support for more Qwen LoRAs#12581
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
|
@bot /style |
|
Style fix runs successfully without any file modified. |
sayakpaul
left a comment
sayakpaul
left a comment
There was a problem hiding this comment.
Thank you! I left some minor yet important nits. Once those are addressed, we can ship!
|
|
||
| state_dict = {convert_key(k): v for k, v in state_dict.items()} | ||
|
|
||
| has_default = any("default." in k for k in state_dict) |
There was a problem hiding this comment.
If it's starting with "default.", then let's be explicit about that:
| has_default = any("default." in k for k in state_dict) | |
| has_default = any(k.startswith("default.") for k in state_dict) |
There was a problem hiding this comment.
"default" isn't the prefix though, e.g.transformer_blocks.0.attn.add_v_proj.lora_A.default.weight
| if has_default: | ||
| state_dict = {k.replace("default.", ""): v for k, v in state_dict.items()} |
There was a problem hiding this comment.
So, that it's done as intended:
| if has_default: | |
| state_dict = {k.replace("default.", ""): v for k, v in state_dict.items()} | |
| if has_default: | |
| state_dict = {k[len("default."):]: v for k, v in state_dict.items()} |
There was a problem hiding this comment.
same comment as here - 'default' is not in the key's prefix, so this won't be the intended behavior in this case
| has_lora_unet = any(k.startswith("lora_unet_") for k in state_dict) | ||
| has_diffusion_model = any(k.startswith("diffusion_model.") for k in state_dict) | ||
| if has_alphas_in_sd or has_lora_unet or has_diffusion_model: | ||
| has_default = any("default." in k for k in state_dict) |
There was a problem hiding this comment.
same comment as #12581 (comment) - 'default' is not in the key's prefix, so this won't be the intended behavior in this case
3 participants
add support for dx8152's Qwen loras (e.g. https://huggingface.co/dx8152/Qwen-Edit-2509-Multiple-angles)