High and Low noise Loras

[oscarchuncha]

As per recent developments, many lora's are being released in 2 parts High noise lora and low noise Lora. I feel the model should implement this as i feel going forward this will be the trend for wan. Many thanks for what you do. Thumbs up and cheers

[Reevoy24]

wan2.2 allready uses both the high and the low loras.

"With models like Wan 2.2 that uses internally two diffusion models (High noise / Low Noise) you can specify which Loras you want to be applied for a specific phase by separating each phase with a ";".

For instance, if you want to disable a lora for phase High Noise and enables it only for phase Low Noise:

0;1
Also with Wan 2.2, if you have two loras and you want the first one to be applied only during the High noise and the second one during the Low noise phase:

1;0 0;1
As usual, you can use any float for of multiplier and have a multiplier varries throughout one phase for one Lora:

0.9,0.8;1.2,1.1,1
In this example multiplier 0.9 and 0.8 will be used during the High Noise phase and 1.2, 1.1 and 1 during the Low Noise phase.

Here is another example for two loras:

0.9,0.8;1.2,1.1,1
0.5;0,0.7
Note that the syntax for multipliers can also be used in a Finetune model definition file (except that each multiplier definition is a string in a json list)"

[feeziwater]

For instance, if you want to disable a lora for phase High Noise and enables it only for phase Low Noise:

0;1

Have you actually tested this to confirm it works as expected?

I've tried multiple times with the same seed and settings, but the output with 0;1 is identical to the one generated without the LoRA at all. Is there any difference in your results?

[Reevoy24]

@feeziwater I've always done it this way: I've used both the high and low LoRa versions, for example, 0.8;1.0
and that's actually worked well so far.

[feeziwater]

@Reevoy24
I mean, under the same settings,
is there a difference between 0.8;1.0 and 0.8;0?
I'm asking because for me, it generates the same output.
I'm trying to clarify if it's just me or if the code is not working as intended.

[Reevoy24]

@feeziwater Yes, you may be right that something isn't implemented quite correctly. 0.8;1.0 and 0.8;0 produce with the same settings and seed the same video.

[Reevoy24]

i checked the code and found out that the Problem could be that:
Two-phase LoRA multipliers were parsed, but Phase 2 often didn’t take effect because only the first diffusion model received the LoRA schedule. I applyed the schedule to both diffusion models. Now 0.8;1.0 vs 0.8;0 yields different results as intended.

but i need more time to test if it really fixes the issue or maybe @deepbeepmeep can check if I am right with my assumption.

[feeziwater]

Now it's fixed!

[Tophness]

Assuming fixed with 51341ac

[deepbeepmeep]

@Reevoy24 thx for spotting this, silly me, forgot one line...