You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
ImageConditioningInput supports explicit latent-replacement and keyframe-guide bindings. On video CLIs, pass --image PATH FRAME_IDX STRENGTH replace|guide [CRF]; omit the binding keyword for the prior automatic binding (frame 0 replaces, later frames guide). Replacement frame indices are expressed in pixel space and must be 0 or a causal latent boundary (1 + 8 * k).
TI2VidTwoStagesPipeline supports keyframe interpolation through ImageConditioningInput.guide() bindings, or --image … guide on the CLI. Set refine_audio=True or pass --refine-audio to preserve the former dedicated
pipeline's stage-2 audio refinement; the default preserves stage-1 audio.
ImageConditioner.load_conditionings(images, color_space=None) resolves each image conditioning's CRF (like resolve_crf), decodes each distinct still once (SDR, or EXR with color_space) and returns LoadedImageConditionings: the decoded still plus the input's binding and strength.
MGPUController(cpu_threads_per_rank=8) caps each rank's intra-op CPU threads (torch.set_num_threads) so the ranks' small host tensor ops do not stall on many-core hosts. Pass None to keep torch's default.
LTX_MGPU_DECODE_RECEIVER_TIMEOUT_SECONDS sets how long a multi-GPU video decode waits for a rank that stopped streaming before it gives up (default 60).
Changed
Multi-GPU pipelines encode prompts faster with the default Accelerate-sharded Gemma: the other ranks now wait on the host until the source rank's Gemma has finished on every GPU, instead of spinning in an NCCL receive on a GPU Gemma is still using. BroadcastCoordinator moved from ltx_pipelines.multigpu._broadcast to ltx_core.multigpu.broadcast, and BroadcastGemmaWrapper (and so AccelerateGemmaWrapper) takes an optional coordinator. BroadcastCoordinator.wait_for_signal_change now consumes exactly one signal per call (a worker that polls late no longer skips a signal) and takes an optional timeout.
ImageConditioningInput now takes an ImageReplace or ImageGuidebinding instead of a frame_idx. Replace existing ImageConditioningInput(path, frame_idx, strength, crf) calls with ImageConditioningInput.at(path, frame_idx, strength, crf) to preserve the previous frame-0 replacement and later-frame guidance behavior; use .replace(path, frame_idx, strength, crf) or .guide(path, frame_idx, strength, crf) to select a binding explicitly. Both bindings use pixel-frame indices.
image_conditionings() and image_conditionings_for_chunk() take LoadedImageConditionings (from ImageConditioner.load_conditionings) instead of ImageConditioningInputs, and no longer take color_space.
Performance improvements, in collaboration with doubleAI (https://doubleai.com): faster transformer passes (especially under multi-GPU sequence parallelism and with perturbation guidance) and LoRA switches, denoising samplers without per-step CPU synchronization, a DFR spatial epilogue (--spatial-upscalings 2) on fewer tiles (at most 1024 px per side, overlapping by 64 px, instead of a 2x2 first step and 4x4 at a 320 px overlap) that multiplies its long video projections in fp8 on GPUs with fp8 GEMMs (compute capability 8.9+), carry keyframes rebuilt on the rank that decodes them, a multi-GPU video decode that streams to MP4 encoding as it decodes, faster diffusion VAE decoding with lower reserved GPU memory, a faster-compiling --diffvae-optimization blackwell_dsl kernel, lower GPU memory during MP4 encoding, faster image conditioning, and faster stage starts with a warm model registry. Output is not bit-identical to previous releases, mainly because of the DFR spatial epilogue's new tiles and fp8 GEMMs; diffusion VAE decodes also differ by rounding, and under multi-GPU sequence parallelism audio cross-attention no longer attends to the video sequence's padding tokens.
Removed
ImageConditioningInput.frame_idx, .assert_in_clip(), .map_frame_index() and .to_item(); use the same members on the LoadedImageConditioning returned by ImageConditioner.load_conditionings, or input.binding.frame_idx.
KeyframeInterpolationPipeline and the ltx_pipelines.keyframe_interpolation CLI; use TI2VidTwoStagesPipeline with ImageConditioningInput.guide() and refine_audio=True, or pass --image … guide with --refine-audio.
Fixed
DiffVAE 1-tile decode no longer produces gray frame tails from frame 128 on long 1080p clips under torch < 2.13. CUDA RMSNorm wrapped flattened offsets at 2^32 elements; last-dim norms are now T-chunked.
Compiling a transformer that uses blockwise FP8/FP6 quantization with seq_dim_dynamic no longer pins the sequence length. Previously each new length recompiled the blocks, and a batch size divisible by 4 failed with a ConstraintViolationError.
A warm multi-GPU DFR job (a second or later request in one process) could fail with "Cannot fit a DiffVAE decode tile". Cold and warm runs and every rank now plan the same decode tiles, and ranks with unequal GPUs no longer hang. --diffvae-optimization blackwell_dsl keyframe decodes plan smaller tiles.
Diffusion VAE decode with --diffvae-optimization blackwell_dsl could hang indefinitely on datacenter Blackwell: the fused stage-5 attention kernel re-armed its per-tile barriers while some warps were still waiting on them.