Skip to content

v1.2.0 - merge default, fast kernels, bypass stacking fix

Choose a tag to compare

@Saganaki22 Saganaki22 released this 04 Sep 03:07
· 35 commits to main since this release

lora_mode now defaults to merge — use merge, especially on 8-step DMD checkpoints (stage-dmd-*). Measured on pruned int8 bases: bypass applies the same adapter math, but each module's delta carries bf16 rounding noise instead of being baked into the weights. Blocks 0-33 stay bit-identical to merge, the deep blocks amplify that noise to ~10% of feature magnitude, and bypass renders of the 8-step model come out degraded; a coherent same-size perturbation (strength 1.016) renders clean, so it is specifically the off-manifold rounding noise. This supersedes v1.1.2's advice to prefer bypass on pruned bases.

Also in v1.2.0:

  • Fixed: bypass hook stacking. Re-running an Apply-VDN node (e.g. after flipping lora_mode) stacked another full set of bypass hooks on the shared model, doubling the LoRA delta per rerun (measured exactly 2.00x) and progressively degrading output. Live hook sets are now tracked on the shared inner model and ejected before re-injection.
  • fast_kernels (Advanced node): torch.compiles the linear branch's hot spots (RMSNorm+gate epilogue, state gather, frame-major q store) into single kernels. Same math, eager fallback, first run compiles.
  • vdn_scaled delta backend registered alongside the scan backend.
  • ~3 GiB less VRAM pressure: dead q/k/v projections are freed before the gate/out-projection/branch readout (upstream parity).
  • READMEs (EN/ZH): lora_mode guidance rewrite; note that both fl2v (fl2va) and ref2v (ref2va) bases are tested and working.
  • Tests: fused-epilogue parity, vdn_scaled coverage, scoped window-dispatch stub, folder-agnostic bypass test. 10/10 green.