Warn and skip transformer LoRA loading when the transformer is absent (#13487) - #13945
Warn and skip transformer LoRA loading when the transformer is absent (#13487)#13945archievi wants to merge 4 commits into
Conversation
…huggingface#13487) Loading a transformer-only LoRA onto a modular sub-pipeline that does not contain the transformer (for example a text-encoder-only sub-pipeline) raised "AttributeError: ... object has no attribute 'transformer'". Following the maintainer suggestion on the issue, load_lora_weights now resolves the transformer with a default of None and, when it is missing, logs a warning and skips loading the transformer LoRA layers instead of raising. The change is made at the copied-from source (CogVideoXLoraLoaderMixin.load_lora_weights) and propagated to the copies with make fix-copies. Adds a regression test.
sayakpaul
left a comment
sayakpaul
left a comment
There was a problem hiding this comment.
Thanks! I left some comments.
| transformer = self.transformer if hasattr(self, "transformer") else getattr(self, self.transformer_name, None) | ||
| if transformer is None: | ||
| logger.warning( | ||
| f"No `{self.transformer_name}` module was found in {self.__class__.__name__}, so no LoRA weights will be " | ||
| "loaded into the transformer. This can happen when loading LoRA weights into a modular pipeline that " | ||
| "does not contain the transformer (for example, a text-encoder-only sub-pipeline)." | ||
| ) |
There was a problem hiding this comment.
This is not specific to a modular pipeline, though.
There was a problem hiding this comment.
Good call — reworded to drop the modular framing. Digging in, I realized the getattr(self, self.transformer_name) crash was not limited to the copied-from group: SD3, Flux, Wan (+SkyReelsV2 via copies), LTX2, Helios and Anima each define their own load_lora_weights and still raised. Extended warn-and-skip to all of them. For the loaders that also load text-encoder LoRAs (SD3/Flux/Anima) it skips only the transformer part and still loads the text-encoder layers rather than returning early.
|
|
||
|
|
||
| @require_peft_backend | ||
| class Flux2LoRAModularPipelineTests(unittest.TestCase): |
There was a problem hiding this comment.
It should be added to
Line 123 in 9b249df
There was a problem hiding this comment.
Moved it — the regression test now lives in PeftLoraLoaderMixinTests (tests/lora/utils.py) as test_load_lora_weights_without_transformer_warns_and_skips, so it runs for every transformer-based loader (skipped for UNet pipelines). It builds the dummy pipeline, nulls out the transformer to emulate a sub-pipeline without one, and asserts we warn and skip instead of raising. Dropped the Flux2-specific class.
Address review on huggingface#13945: - Reword the warning so it is no longer framed as modular-pipeline specific (the underlying `getattr(self, self.transformer_name)` crash applies to any (sub-)pipeline that lacks a transformer component). - Extend warn-and-skip to the transformer loaders that define their own `load_lora_weights` and were not covered by the copied-from source: SD3, Flux, Wan (+SkyReelsV2 via copies), LTX2, Helios and Anima. For loaders that also load text-encoder LoRAs (SD3/Flux/Anima), only the transformer part is skipped so text-encoder layers still load. - Move the regression test for huggingface#13487 from a Flux2-specific class into the shared `PeftLoraLoaderMixinTests`, so it runs for every transformer-based loader (skipped for UNet pipelines). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
github-actions
Bot
added
size/L
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
…missing-transformer-13487
Merged main brought in Krea2LoraLoaderMixin and Ideogram4LoraLoaderMixin, which are `# Copied from` CogVideoXLoraLoaderMixin.load_lora_weights. Ran make fix-copies so they carry the same missing-transformer warn-and-skip guard, fixing check_repository_consistency.
|
Rebased on |
3 participants
What does this PR do?
Loading a transformer-only LoRA onto a Modular sub-pipeline that does not contain the transformer (for example a text-encoder-only
text_pipe) currently crashes:This happens because
load_lora_weightsresolves the transformer withgetattr(self, self.transformer_name), which raises when the (sub-)pipeline has no transformer component.Following @yiyixuxu's suggestion on the issue ("maybe warn and skip is better"),
load_lora_weightsnow resolves the transformer with aNonedefault and, when no transformer is present, logs a warning and skips loading the transformer LoRA layers instead of raising.The change is made at the copied-from source (
CogVideoXLoraLoaderMixin.load_lora_weights) and propagated to its 14 copies withmake fix-copies, so the behavior is consistent across the transformer LoRA loaders (Flux2, Flux, SD3, CogView4, etc.). Normal pipelines that do have a transformer are unaffected.Fixes #13487
Before submitting
.ai/? I read and applied.ai/AGENTS.md(copied-code convention) and.ai/modular.md; the fix is made at the copied-from source and propagated withmake fix-copiesrather than editing individual copies..ai/review-rules.md? No ephemeral context, correctness-focused, no dead code.Flux2LoRAModularPipelineTestsintests/lora/test_lora_layers_flux2.py.Verification (run locally)
AttributeErrorwhen the fix is reverted (regression-proven).python utils/check_copies.pypasses (copies consistent aftermake fix-copies).ruff checkandruff format --checkare clean on the changed files.Who can review?
@yiyixuxu @sayakpaul @DN6