Skip to content

v0.6.0 — Gradient guidance, multiscale sampling, U-Net acceleration

Latest

Choose a tag to compare

@Sxela Sxela released this 07 Aug 18:18

Gradient guidance is back on the render path, alongside multiscale sampling and two
opt-in U-Net speedups.

Gradient guidance

WarpFusion's guidance now steers sampling toward a temporal target, independently of the
img2img init (which stays the warp + consistency composite):

  • Pixel guidanceinit_scale (try 1000), an LPIPS + MSE loss against the guidance
    target in pixel space.
  • Latent guidanceinit_latent_scale (try 100), an MSE loss in latent space.
  • guidance_mode picks the target: previous warped frame, with or without the
    consistency mask, the raw init, and so on.
  • clamp_grad / clamp_max clamp the gradient by RMS so a large scale does not blow
    the frame out.
  • guidance_add_noise / guidance_use_start_code noise the target to the sampler's
    current sigma, reusing one noise tensor and preserving the notebook's RNG order.

LPIPS moved out of the optional [sched] extra into the core dependencies — core guidance
needs it now. [sched] still exists so nothing breaks, but it is empty.

Multiscale sampling

sampler_scale_schedule maps a sampling step to a percentage of the final latent
resolution. Early steps can evaluate the whole frame cheaply and later steps run at full
resolution. sampler_scale_min_size sets a pixel floor for the longest side, so coarse
passes stay at or above the model's trained resolution (512 for SD1.5).

Below 100% the U-Net sees the whole downscaled frame and tiling is deliberately disabled;
at 100% the normal tiled path takes over. The sampler latent itself is always full
resolution.

U-Net acceleration (opt-in)

unet_cache offers two strategies, both resetting their state every frame:

  • DeepCache — reuses the expensive inner U-Net branch on intermediate denoiser
    evaluations, every unet_cache_interval steps.
  • First Block Cache — reuses a previous output when the first encoder block moves less
    than unet_cache_threshold relative to the last one.

compile_unet separately torch.compiles the unified U-Net, the active ControlNets, and
the VAE encode/decode paths. It is opt-in because every new multiscale/tile/VAE shape may
trigger another graph compile; Inductor artifacts persist under .vibewarp_cache.

Per-part pose detection

OpenPose and DWPose now expose pose_include_body, pose_include_hand, and
pose_include_face, controlling which keypoint sets are detected and drawn.

Fixes

  • SDXL text encoder on newer open_clip builds whose attention modules are
    batch-first — the unconditional NLD→LND permute produced silently wrong conditioning.
  • IP-Adapter composition precise weight type, matching the current ComfyUI
    IPAdapter Plus layer map.
  • The UI launchers (run-ui.bat / run-ui.sh) now reinstall dependencies when an
    existing environment predates a required package, such as torchsde for the DPM++ SDE
    samplers.

As always: parity-validated coverage and what is implemented-but-unverified are tracked in
docs/roadmap.md. The features
in this release are covered by the CPU test suite but have not been through GPU parity
validation against the notebook.

Full changelog: v0.5.0...v0.6.0