Skip to content

v1.5.0

Latest

Choose a tag to compare

@lt-research-bot lt-research-bot released this 18 Oct 17:41

Added

  • ImageConditioningInput supports explicit latent-replacement and keyframe-guide bindings. On video CLIs, pass --image PATH FRAME_IDX STRENGTH replace|guide [CRF]; omit the binding keyword for the prior automatic binding (frame 0 replaces, later frames guide). Replacement frame indices are expressed in pixel space and must be 0 or a causal latent boundary (1 + 8 * k).
  • TI2VidTwoStagesPipeline supports keyframe interpolation through ImageConditioningInput.guide() bindings, or
    --image … guide on the CLI. Set refine_audio=True or pass --refine-audio to preserve the former dedicated
    pipeline's stage-2 audio refinement; the default preserves stage-1 audio.
  • ImageConditioner.load_conditionings(images, color_space=None) resolves each image conditioning's CRF (like resolve_crf), decodes each distinct still once (SDR, or EXR with color_space) and returns LoadedImageConditionings: the decoded still plus the input's binding and strength.
  • MGPUController(cpu_threads_per_rank=8) caps each rank's intra-op CPU threads (torch.set_num_threads) so the ranks' small host tensor ops do not stall on many-core hosts. Pass None to keep torch's default.
  • LTX_MGPU_DECODE_RECEIVER_TIMEOUT_SECONDS sets how long a multi-GPU video decode waits for a rank that stopped streaming before it gives up (default 60).

Changed

  • Multi-GPU pipelines encode prompts faster with the default Accelerate-sharded Gemma: the other ranks now wait on the host until the source rank's Gemma has finished on every GPU, instead of spinning in an NCCL receive on a GPU Gemma is still using. BroadcastCoordinator moved from ltx_pipelines.multigpu._broadcast to ltx_core.multigpu.broadcast, and BroadcastGemmaWrapper (and so AccelerateGemmaWrapper) takes an optional coordinator. BroadcastCoordinator.wait_for_signal_change now consumes exactly one signal per call (a worker that polls late no longer skips a signal) and takes an optional timeout.
  • ImageConditioningInput now takes an ImageReplace or ImageGuide binding instead of a frame_idx. Replace existing ImageConditioningInput(path, frame_idx, strength, crf) calls with ImageConditioningInput.at(path, frame_idx, strength, crf) to preserve the previous frame-0 replacement and later-frame guidance behavior; use .replace(path, frame_idx, strength, crf) or .guide(path, frame_idx, strength, crf) to select a binding explicitly. Both bindings use pixel-frame indices.
  • image_conditionings() and image_conditionings_for_chunk() take LoadedImageConditionings (from ImageConditioner.load_conditionings) instead of ImageConditioningInputs, and no longer take color_space.
  • Performance improvements, in collaboration with doubleAI (https://doubleai.com): faster transformer passes (especially under multi-GPU sequence parallelism and with perturbation guidance) and LoRA switches, denoising samplers without per-step CPU synchronization, a DFR spatial epilogue (--spatial-upscalings 2) on fewer tiles (at most 1024 px per side, overlapping by 64 px, instead of a 2x2 first step and 4x4 at a 320 px overlap) that multiplies its long video projections in fp8 on GPUs with fp8 GEMMs (compute capability 8.9+), carry keyframes rebuilt on the rank that decodes them, a multi-GPU video decode that streams to MP4 encoding as it decodes, faster diffusion VAE decoding with lower reserved GPU memory, a faster-compiling --diffvae-optimization blackwell_dsl kernel, lower GPU memory during MP4 encoding, faster image conditioning, and faster stage starts with a warm model registry. Output is not bit-identical to previous releases, mainly because of the DFR spatial epilogue's new tiles and fp8 GEMMs; diffusion VAE decodes also differ by rounding, and under multi-GPU sequence parallelism audio cross-attention no longer attends to the video sequence's padding tokens.

Removed

  • ImageConditioningInput.frame_idx, .assert_in_clip(), .map_frame_index() and .to_item(); use the same members on the LoadedImageConditioning returned by ImageConditioner.load_conditionings, or input.binding.frame_idx.
  • KeyframeInterpolationPipeline and the ltx_pipelines.keyframe_interpolation CLI; use TI2VidTwoStagesPipeline with ImageConditioningInput.guide() and refine_audio=True, or pass --image … guide with --refine-audio.

Fixed

  • DiffVAE 1-tile decode no longer produces gray frame tails from frame 128 on long 1080p clips under torch < 2.13. CUDA RMSNorm wrapped flattened offsets at 2^32 elements; last-dim norms are now T-chunked.
  • Compiling a transformer that uses blockwise FP8/FP6 quantization with seq_dim_dynamic no longer pins the sequence length. Previously each new length recompiled the blocks, and a batch size divisible by 4 failed with a ConstraintViolationError.
  • A warm multi-GPU DFR job (a second or later request in one process) could fail with "Cannot fit a DiffVAE decode tile". Cold and warm runs and every rank now plan the same decode tiles, and ranks with unequal GPUs no longer hang. --diffvae-optimization blackwell_dsl keyframe decodes plan smaller tiles.
  • Diffusion VAE decode with --diffvae-optimization blackwell_dsl could hang indefinitely on datacenter Blackwell: the fused stage-5 attention kernel re-armed its per-tile barriers while some warps were still waiting on them.