Skip to content

Untiled LTX VAE decode when it fits, drop per-tile gc - #38

Merged
TroyHernandez merged 3 commits into
mainfrom
perf/ltx-decode
Jul 20, 2026
Merged

Untiled LTX VAE decode when it fits, drop per-tile gc#38
TroyHernandez merged 3 commits into
mainfrom
perf/ltx-decode

Conversation

@TroyHernandez

Copy link
Copy Markdown
Contributor

Summary

The video decode phase measured 12.3 s in-render. Profiling it in isolation (VAE only, synthetic latents at the 768x512x49 shape): the tiled decode was 4.35 s, of which 1.89 s was the explicit per-tile gc(), and a single untiled forward is 2.67 s. Tiling exists to bound activation VRAM, not for correctness — and at decode time the transformer is phase-offloaded, so that VRAM is actually free.

  • decode() now runs one full-latent forward when the estimated activation cost fits the card. Cost is linear in output pixel-frames: ~1 GB + ~350 B/pxf, calibrated across 2.5M–19.3M pixel-frames (three sizes, linear to within noise); the rule uses 360 B/pxf plus 15% headroom against total-minus-allocated. options(diffuseR.vae_untiled = TRUE/FALSE) forces either path; "auto" applies only on CUDA and CPU keeps the tiled behavior. 1024x576x121 (~26 GB estimate) correctly stays tiled on a 16 GB card.
  • The per-tile explicit gc() in both tiled paths is gone. It guarded against an allocator-callback storm that no longer occurs with the gc gates live (Activate the LTX allocator gates and pinned staging #37): measured 3.8 s vs 4.2 s per decode, storm-free, VRAM peak unchanged.

Measurements

768x512x49, NF4, pinned staging, seed 42, RTX 5060 Ti:

before (post-#37) this PR
video decode phase 11.4–13.2 s 1.7–1.8 s
warm render 54.0–54.3 s 44.2–44.5 s

The week's cumulative arc for this render: 89–96 s → 73 s (byte-LUT #35) → 54 s (gates + pinning #37) → 44 s.

Validation

  • Full local suite: 892 results, all green, including 3 new dispatch tests (forced-untiled output identical to the untiled reference, forced-tiled identical to the tiled path, auto-on-CPU stays tiled).
  • Calibration note: a first calibration attempt read a flat 13.0 GB peak at every size — that was the loader's pool pre-warm polluting the cumulative peak stat, dodged by loading on CPU and moving the VAE manually.

The video decode phase measured 12.3 s in-render. Profiling (VAE
only, synthetic latents): the tiled decode itself was 4.35 s of which
1.89 s was the explicit per-tile gc(), and a single untiled forward is
2.67 s - tiling exists to bound activation VRAM, not for correctness,
and at decode time the transformer is phase-offloaded so the VRAM is
actually free.

decode() now runs one full-latent forward when the estimated
activation cost fits the card: cost is linear in output pixel-frames
(~1 GB + ~350 B/pxf measured across 2.5M-19.3M pxf; the rule uses
360 B/pxf + 15% headroom against total minus allocated).
options(diffuseR.vae_untiled = TRUE/FALSE) forces either path; auto
applies only on CUDA, CPU keeps tiled behavior. The per-tile explicit
gc() goes away: with the allocator gates live (#37) the dead-handle
storm it guarded against no longer occurs (measured 3.8 s vs 4.2 s,
storm-free, VRAM peak unchanged).

In-render (768x512x49 NF4, pinned, seed 42): video decode phase
12.3 -> 1.7-1.8 s; warm render 54 -> 44.2-44.5 s. Tests: dispatch
parity for forced-untiled vs reference, forced-tiled vs tiled, and
auto-on-CPU staying tiled.
@TroyHernandez
TroyHernandez merged commit 4c08448 into main Jul 20, 2026
2 checks passed
@TroyHernandez
TroyHernandez deleted the perf/ltx-decode branch July 20, 2026 20:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant