Repository navigation
loraspectra: Fast Rust CLI to audit, compress, convert (PEFT/Kohya/Diffusers), and orthogonally merge Diffusion LoRA/DoRA .safetensors #15255
SalyyS1
started this conversation in
Show and tell
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Hi everyone,
When training or combining Flux.1, Flux.2, SDXL, Wan, or LTX-Video LoRA/DoRA checkpoints in Diffusers, three practical problems come up constantly:
.lora_A.weight), Kohya/ComfyUI (.lora_down.weight,.dora_scale), Diffusers (.lora.down.weight), and wrapper prefixes (model.diffusion_model.,transformer.,base_model.model.).double_blocksandsingle_blockswastes 60-75% of parameters on Marchenko-Pastur noise tail singular values.We built loraspectra, a zero-dependency Rust CLI that runs exact$O(r^3)$ QR-projected SVD directly on
.safetensorsfiles in milliseconds on CPU.What it does for Diffusion LoRAs
loraspectra audit <file.safetensors>: Automatically detects Flux.1 / Flux.2 DiT, SDXL, SD1.5, or VLM architectures from tensor shapes alone, reports per-layer effective rank (R_EFF), 99% energy rank (K_99%), and flags dead or over-parameterized blocks.loraspectra compress: Truncates each layer to its 99% spectral energy rank (--energy 0.99) or Marchenko-Pastur threshold (--policy mp), casts tof16/bf16, and converts between--format diffusers,kohya, andpeft(including DoRA.dora_scale/.lora_magnitude_vectorand.alphakeys).loraspectra interference&loraspectra merge --mode ortho-svd: Measures principal subspace cosine overlap between 2+ LoRAs and merges.safetensorsfile.loraspectra extract: Extracts a clean low-rank LoRA directly fromW_tuned - W_basebetween two dense.safetensorscheckpoints (normalizing ComfyUI/SRPOmodel.diffusion_model.vs Diffusers prefixes automatically).Repository & prebuilt binaries: https://github.com/SalyyS1/loraspectra
If this helps your Diffusers LoRA workflows, please consider starring SalyyS1/loraspectra on GitHub!
All reactions