loraspectra v1.1.0

loraspectra v1.1.0

Highlights

• DoRA Directional vs Magnitude Spectral Diagnostics (loraspectra audit / LS019-LS020): Computes exact per-output-channel directional norms ||(B * A)_{i,:}||_2 via the r x r Gram matrix M_A = A * A^T in O(r^2 * (d_in + d_out)) time without dense allocations, measuring channel magnitude dispersion (mag_cv, max/min), dead directional rows, and magnitude-direction Pearson correlation.
• OASIS SARIF v2.1.0 Rules LS019 & LS020: Detects DoRA magnitude shape mismatches, zero/negative/non-finite channel scales, and partial DoRA coverage (LS019), plus extreme >50x channel dispersion and directional B * A stagnation (LS020).
• DoRA Differential Drift & Coupling (loraspectra diff): Decomposes checkpoint drift into low-rank directional Frobenius/subspace drift and 1D lora_magnitude_vector L2/relative drift, reporting per-channel coupling between |m_B - m_A| and ||(Delta W_B - Delta W_A)_{i,:}||_2.
• 1D Safetensors Preservation, --exclude Filtering & Scale-Invariant DoRA Merge: Preserves 1D [d_out] shapes across .lora_magnitude_vector, .lora_magnitude_vector.weight, .dora_scale, and .magnitude suffixes, prunes excluded layer magnitude vectors during compress --exclude, and convex-combines DoRA scales in merge.

loraspectra v1.0.0 - Global Budget Rank Allocator & Scale-Preserving PEFT Config Export

LoRASpectra v1.0.0

Zero-dependency Rust engine, CLI, and client-side Web Inspector for spectral SVD auditing, silent SFT bug diagnostics, budget-optimal heterogeneous rank compression, and subspace-orthogonal merging of HuggingFace .safetensors LoRA/DoRA adapters.

Highlights in v1.0.0

1. Global Budget-Optimal Heterogeneous Rank Allocator (loraspectra compress / extract):

   • Allocates heterogeneous per-layer ranks k_l in [min_rank, max_rank] across the full adapter under --target-rank <avg_r>, --budget-ratio <0..1>, or exact --max-bytes <size> constraints using O(L r log L) marginal spectral energy-per-byte water-filling ((s_l * sigma_{l,j})^2 / c_l).
   • Outperforms uniform fixed-rank truncation at identical byte budgets by shifting rank capacity from rank-collapsed projections toward high-entropy layers.

2. Scale-Preserving HuggingFace PEFT Config Export (--peft-config / --out-dir):

   • Emits complete, drop-in adapter_config.json files with exact dot-boundary-safe rank_pattern and alpha_pattern dictionaries (alpha_l = s_l * k_l for standard LoRA and alpha_l = s_l * sqrt(k_l) for rsLoRA).
   • Preserves upstream base_model_name_or_path, task_type, target_modules (including "all-linear" or inferred leaf projections), use_dora, and modules_to_save so PeftModel.from_pretrained loads heterogeneous compressed adapters with identical forward-pass scaling.

3. Expanded Test Coverage & CI Verification:

   • Verified across 34/34 unit tests covering standard and rsLoRA scale-preserving roundtrips, exact --max-bytes and dtype-aware water-filling, Diffusers/OneTrainer key mapping, SARIF v2.1.0 LS001-LS018 schema validation, and dense checkpoint extraction.

loraspectra v0.9.0 - Multi-Adapter Merge Conflict Analyzer & Browser Header Inspector

LoRASpectra v0.9.0

Fast spectral SVD auditor, rank-collapse checker, multi-adapter merge interference analyzer, and silent SFT bug diagnostics for HuggingFace .safetensors LoRA adapters.

What is New in v0.9.0

1. Multi-Adapter Merge Conflict Analyzer ():

   • Analyzes two or more LoRA adapters for subspace collisions, sign cancellations, collinear inflation, and extreme alpha scaling prior to merging with TIES, DARE, or linear algorithms.
   • Computes principal subspace angles, Grassmannian overlap, and Frobenius inner product alignment across target modules in O(r^3) time without dense matrix allocations.

2. OASIS SARIF Rules LS015 through LS018:

   • Adds automated SARIF diagnostic reporting for merge conflicts:
     • LS015: Severe destructive subspace cancellation.
     • LS016: Collinear magnitude inflation.
     • LS017: Severe alpha scaling mismatch across adapters.
     • LS018: Overcomplete combined rank relative to target layer dimensions.

3. Interactive Browser Header Inspector:

   • Enhances the client-side Web Inspector (https://salyys1.github.io/loraspectra/) with zero-upload .safetensors header parsing, layer architecture introspection, and instant conflict detection directly in the browser.

Verification

• 29/29 unit tests passed (cargo test).
• Zero external runtime crate dependencies.

loraspectra v0.8.0 - OASIS SARIF v2.1.0 Code Scanning & Checkpoint Differential Drift

LoRASpectra v0.8.0

Fast spectral SVD auditor, rank-collapse checker, and silent SFT bug diagnostics for HuggingFace .safetensors LoRA adapters.

What is New in v0.8.0

1. OASIS SARIF v2.1.0 Code Scanning Export (--sarif <file>):

   • Native OASIS SARIF v2.1.0 standard JSON output mapping rules LS001 through LS014.
   • Flags critical fine-tuning failures including dead LoRA layers (frozen gradients), zero-norm token embedding rows (untrained special tokens), collapsed rank, and alpha/DoRA scaling anomalies.
   • Plugs directly into GitHub Code Scanning via github/codeql-action/upload-sarif.

2. LoRA Checkpoint Differential Drift & Subspace Overlap (loraspectra diff):

   • Compares two fine-tuning checkpoints across training steps or hyperparameter sweeps in O(r^2 (d_in + d_out)) time with zero dense matrix allocations.
   • Reports layer-by-layer Frobenius drift, relative drift, and Grassmannian subspace principal cosine similarity in human-readable tables or machine-readable JSON (--json).

3. Composite GitHub Action (action.yml):

   • Reusable GitHub Action (SalyyS1/loraspectra@v0.8.0) for auditing LoRA checkpoints and failing CI builds on critical adapter defects.

Verification

• 24/24 unit tests passed (cargo test).
• Zero external runtime crate dependencies.

loraspectra v0.7.0 - Silent SFT Bug Diagnostics & HuggingFace Model Card Generator

Highlights in v0.7.0

• Adapter Integrity & Silent Fine-Tuning Bug Diagnostics (loraspectra audit [--json]):
  • Detects zero-norm / dead token rows in saved dense embedding and unembedding tensors (embed_tokens, wte, lm_head) and LoRA adapters targeting them (lora_embedding_A/B and lm_head lora_B), catching silent SFT bugs where newly added special tokens such as <tool_call> never receive gradient updates.
  • Detects dead lora_B rank columns, completely zero-norm LoRA layers, missing lora_alpha in adapter_config.json when .alpha scalar tensors are absent, partial or mismatched .alpha coverage across layers, extreme alpha/r ratios (< 0.1 under-scaling or > 16.0 norm explosion), and DoRA magnitude vector shape/value anomalies.
• HuggingFace Model Card Generator (loraspectra card <adapter_or_hf_uri>):
  • Emits a ready-to-paste Markdown Spectral Audit Report for HuggingFace README.md Model Cards with static shields.io badges, spectral & parameter summary metrics, and top capacity-bottleneck vs over-parameterized target module tables.
• PEFT nn.Embedding LoRA Support:
  • Native extraction and inspection of .lora_embedding_A / .lora_embedding_B adapter pairs.

v0.6.0 - Remote HuggingFace hf:// Inspection, Interference Merge Advisor & Browser .safetensors SVD Inspector

Highlights in v0.6.0

• Zero-Download-Friction Remote HuggingFace Hub & URL Support (hf://): Inspect, audit, compress, and merge .safetensors LoRA checkpoints directly via hf://<org>/<repo>[/<file.safetensors>] or https://... URLs with automatic sibling discovery, $HF_TOKEN support, and local caching.
• Automated Subspace Interference Merge Advisor (loraspectra interference --json): Analyzes global and block-level (attn vs mlp/ff) principal subspace cosines and Frobenius sign cancellation to recommend exact --mode (linear, ties, or ortho-svd), --rank, and localized --layer-weights commands.
• Interactive 100% Client-Side Browser .safetensors Spectral Inspector: Drop any local .safetensors file (or try built-in Rank-64 PEFT & Rank-16 FLUX demos) at https://salyys1.github.io/loraspectra/ to inspect QR-Core + Jacobi SVD singular value decay, Marchenko-Pastur noise floors, and adaptive energy savings completely offline in your browser.

v0.5.0: DoRA Magnitude Vectors, Dense-to-LoRA Extraction CLI & Cross-Ecosystem Key Conversion

What is New in loraspectra v0.5.0

1. Dense Base-to-Tuned LoRA Extraction (loraspectra extract <base.safetensors> <tuned.safetensors> <output_lora.safetensors>):
   • Computes the dense delta $\Delta W = W_{\text{tuned}} - W_{\text{base}}$ across matching 2D weight matrices and extracts an optimal low-rank LoRA adapter via Eckart-Young energy allocation (--energy 0.99), fixed rank (--rank K), or hyperparameter-free Marchenko-Pastur SGD noise floor pruning (--mp-denoise), optionally emitting a HuggingFace PEFT adapter_config.json (--peft-config).
2. DoRA (lora_magnitude_vector / dora_scale) 1D Tensor Preservation:
   • Reads and writes 1D DoRA magnitude vectors across .safetensors archives during compression and merging (extract_dora_scales, insert_dora_scales), addressing invoke-ai/InvokeAI#10019.
3. Cross-Ecosystem Key Format Conversion (--format peft|kohya|diffusers):
   • Converts adapter tensor keys between HuggingFace PEFT (.lora_A.weight / .lora_B.weight), Kohya-ss / ComfyUI (.lora_down.weight / .lora_up.weight), and Diffusers (.lora.down.weight / .lora.up.weight) during compress, merge, and extract.

loraspectra v0.4.0 — .alpha & rsLoRA Scale Baking, N-Adapter Joint Ortho-SVD Merge, and Checkpoint Inspector

New in v0.4.0

• .alpha Scalar Tensor Preservation & rsLoRA / rank_pattern Scale Baking (--bake-alpha, --rslora, --alpha A): Folds alpha / r or alpha / sqrt(r) directly into (B, A) via sqrt(scale), allowing heterogeneous rank_pattern and rsLoRA adapters to serve with exact scaling on vLLM, llama.cpp, ComfyUI, and InvokeAI.
• N-Adapter (N >= 2) Joint Orthogonal Subspace Merge (merge_multiple_adapters): Merge 2, 3, or more .safetensors adapters in a single pass (loraspectra merge a.safetensors b.safetensors c.safetensors out.safetensors --weights w1:w2:w3) with lossless intermediate subspace orthogonalization.
• Checkpoint Architecture Reverse-Engineering (loraspectra info): Inspects any .safetensors LoRA file without adapter_config.json and reports the naming convention (PEFT, Kohya/ComfyUI, Diffusers, OneTrainer), inferred base model family (Flux.1, SDXL, Llama-3, Qwen2.5), rank distribution, and spectral health.

loraspectra v0.3.0: Marchenko-Pastur Denoising, PEFT rank_pattern Export & Norm-Bounded Merging

Highlights in v0.3.0

• Gavish-Donoho (2014) Marchenko-Pastur Automatic Denoising (--mp-denoise & K_MP): Automatically separates signal singular values from stochastic SGD tail noise without manual rank or energy threshold tuning.
• HuggingFace PEFT rank_pattern & alpha_pattern Export (--peft-config): Generates a valid adapter_config.json when dynamically compressing LoRAs so PeftModel.from_pretrained loads heterogeneous per-layer ranks without shape errors.
• Disjoint Module & Heterogeneous-Rank Merging: Merges adapters even when r1 != r2 per layer or when adapters target partially overlapping module sets.
• Layer-Block Weighting (--layer-weights) & Spectral Norm Bounding (--preserve-norm): Assign custom weight ratios per subsystem (e.g. self_attn=1.0:0.3,mlp=0.4:1.0) and bound merged spectral norms to prevent activation saturation.
• Cross-DType Precision Casting (--dtype f16|bf16|f32): Downcast F32 checkpoints to BF16 or F16 in the same pass as SVD rank compression.

v0.1.0 — QR-Core Spectral SVD Auditor, Compressor & Orthogonal Merger

loraspectra v0.1.0 — Spectral SVD Auditor, Dynamic Rank Compressor & Orthogonal Subspace Merger for .safetensors LoRA

Highlights

• QR-Core SVD Factorization (O(r^2 d + r^3)): Computes exact singular value spectra of (lora_A, lora_B) and Kohya/FLUX (lora_down, lora_up) pairs in F32, F16, and BF16 1,050x–2,680x faster than full dense SVD (1.08e-13 max singular value error vs dense f64 SVD).
• loraspectra audit: Reports per-layer Spectral Norm, Frobenius Norm, Stable Rank (r_stable), von Neumann Spectral Entropy Effective Rank (r_eff), and k_99% energy rank.
• loraspectra compress: Eckart-Young-Mirsky dynamic energy allocation (--energy 0.99) reducing 28-layer Transformer LoRA footprint by -83.8% while retaining 99.19% spectral energy.
• loraspectra interference & loraspectra merge --mode ortho-svd: Grassmannian principal subspace overlap diagnostics and Orthogonal Subspace Projection SVD merging across mismatched adapter ranks.

Prebuilt Binary (Linux x86_64, Zero Dependencies)

curl -sL https://github.com/SalyyS1/loraspectra/releases/download/v0.1.0/loraspectra-v0.1.0-x86_64-unknown-linux-gnu.tar.gz | tar -xz
./loraspectra audit adapter_model.safetensors