fix(ltx2): wire conv_in to correct widths in LTX2VideoUpBlock3d - #14308
Open
Solaris-star wants to merge 1 commit into
Open
fix(ltx2): wire conv_in to correct widths in LTX2VideoUpBlock3d#14308Solaris-star wants to merge 1 commit into
Solaris-star wants to merge 1 commit into
Conversation
Two independent defects in __init__ made any non-nominal decoder_block_out_channels unloadable (huggingface#14307): 1. conv_in projected onto out_channels, but the upsampler that consumes its output expects out_channels * upscale_factor. Fix: project onto out_channels * upscale_factor and gate conv_in creation on in_channels != out_channels * upscale_factor. 2. LTX2VideoDecoder3d passed input_channel = output_channel // upsample_factor[i] to each up block, but the tensor arriving from the previous block is output_channel wide (no division needed). The division produced a fictitious width corresponding to no tensor in the graph. Fix: input_channel = output_channel. Both defects are invisible on released LTX-2 checkpoints because the nominal decoder_block_out_channels happen to make conv_in unnecessary and the division a no-op. They surface immediately with any other decoder width configuration. Adds unit tests covering the conv_in projection, the no-conv_in nominal case, and the no-upsampler case. Fixes huggingface#14307
github-actions
Bot
added
fixes-issue
models
tests
size/M
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #14307
Two independent defects in
LTX2VideoUpBlock3d.__init__made any non-nominaldecoder_block_out_channelsunloadable:1.
conv_inprojected ontoout_channels, but the upsampler expectsout_channels * upscale_factor.The upsampler is built with
in_channels=out_channels * upscale_factor, soconv_inmust project onto that same width. The guard should comparein_channels != out_channels * upscale_factor.2.
LTX2VideoDecoder3ddividedoutput_channelby the current block'supsample_factorto computeinput_channel.The tensor arriving from the previous block is
output_channelwide — no division needed. The division produced a fictitious width corresponding to no tensor in the graph.Both defects are invisible on released LTX-2 checkpoints because the nominal
decoder_block_out_channelshappen to makeconv_inunnecessary and the division a no-op.Tests
Added
tests/models/autoencoders/test_autoencoder_kl_ltx2.pywith 3 tests:test_conv_in_projects_to_upsampler_input_width— non-nominal widths trigger conv_in and forward pass succeedstest_no_conv_in_when_widths_match— nominal case still works without conv_intest_conv_in_with_no_upsampler— no-upsampler case works correctly