🚨 Give mixed-rank LoRAs without alpha keys their intended scale

[apolinario]

get_peft_kwargs takes lora_alpha from the first entry of the rank dict and never revisits it, and with no alpha keys alpha_pattern stays empty. For an adapter with mixed ranks and no alpha keys that means every module whose rank differs from the first key's rank gets an arbitrary scale: PEFT falls back to the global lora_alpha (peft/tuners/lora/model.py, alpha_pattern.get(key, config.lora_alpha)), so the direction of the error depends on state dict key order.

Real case: larryvrh/MiniMax-H3-Turbo-Lora ships 518 tensors, ranks 64 (attention/FFN) and 16 (AdaLN), zero alpha keys, and states W_eff = W + lora_B @ lora_A, alpha = rank. The AdaLN key sorts first, so lora_alpha = 16, r = 64 and all rank 64 modules load at 0.25x. Its pre converted mirror (InstantX/MiniMax-H3-Turbo-Lora-Diffusers) documents a manual network_alphas workaround for exactly this.

The fix mirrors the ranks into the alphas when the checkpoint brings no alpha information, which is the diffusers/PEFT convention (alpha == rank, scale 1.0) and what load_lora_adapter already does for SAI control LoRAs a few lines below. It is gated on the absence of alpha data: mixed rank adapters with a declared alpha keep it (a uniform declared alpha must not fall back to per module ranks), uniform rank adapters are a no-op, and the metadata serialization path is unaffected because it bypasses this function entirely.

🚨 Behavior change for existing mixed rank, no alpha LoRAs: they previously loaded at an order dependent wrong scale and now load at the intended scale 1.0.

Kohya conversion precedent for the invariant: lora_conversion_utils.py folds alpha / rank into the weights with default_alpha = rank, i.e. it also assumes scale 1.0 when alpha is absent.

Tests cover the mixed rank no alpha case (including key order independence), uniform rank, uniform declared alpha, and per module alphas.

[apolinario]

(unsure if we need this test)