feat(LTX25-DFR-PIPELINE): the DFR canvas, the slots it invents, and the loop that still does not run (#986)

[localai-bot]

DFRPipeline's BASE is ported and reachable: --pipeline-kind dfr resolves upstream's padded keyframe canvas, places generated keyframe slots on its segment grid, marks them, reads them back before the trim, and trims the canvas to the frame count the caller asked for. Generated keyframe slots are SERVED on every pipeline, which retires the refusal #920 landed.

The temporal refinement ROUNDS are not ported and are refused by name, so the temporal x2 upsampler is still UNDRIVEN and docs/FEATURES.md still says so. That is the row's headline result and it is stated first, because the row was dispatched expecting the opposite.

Closes nothing. Issue #986 stays open for the rounds loop and the standalone slot decode, both listed under ## Owed. Campaign #644. Spec .agents/specs/ltx25-dfr-pipeline.md. Pin Lightricks/LTX-2 @ fd4ded7f2d88d3da713abcdd4ad41ecc4a9314ca, verified at the local checkout with git rev-parse HEAD before any anchor was read.

What landed

ltx2_dfr.{h,cpp} is dfr_layout.py in full plus the three dfr_pipeline.py helpers that are pure index arithmetic. It is its own translation unit because every one of its failure modes returns a correctly shaped, finite, plausible latent: there is no NaN, no shape disagreement and no magnitude to bound, so the only instrument that can see a defect is an exact index vector. test_ltx2_dfr compares element by element against values the upstream module RETURNED, not against a reading of its source.

ltx2_conditioning gains Ltx2ConditionVideoByGeneratedKeyframeSlots, Ltx2GeneratedKeyframeLayout and Ltx2ExtractGeneratedKeyframes, with Ltx2ClearConditioning extracting BEFORE it trims. Four things there invert or disable something every other appending item does, and each is annotated at its site: the content goes into latent and not clean, the denoise mask is 1 and not 1 - strength, marked is TRUE (upstream's only such call site), and the temporal span is one pixel frame written by hand.

ltx2_video.cpp gains the dfr pipeline kind, the canvas resolution, the per-phase slot conditioning, the slot carry between stages through the spatial upsampler, the trim back to the caller's contract, and the cut of the soundtrack to the picture.

Reach, traced rather than assumed. pipeline_kind is a LOAD knob, so all three surfaces carry it: ltx2-gen --pipeline-kind dfr, the C ABI's vllm_video_model_params.extra_keys / extra_values, and the server's --video-extra pipeline_kind=dfr at launch (server_main.cpp:492-503 parses it, :1317-1325 copies it into vmp.extras). A server started that way renders every /v1/videos request through DFR. This PR originally claimed CLI and ABI only, on the grounds that /v1/videos forwards no engine extras (#928) — true of the per-GENERATION extras num_generated_keyframes and temporal_upsample_rounds, and false of pipeline_kind. Corrected in docs/USAGE.md and here.

The oracle, and why it is not an import

scripts/gen-ltx2-dfr-goldens.py imports and EXECUTES dfr_layout. It does not import dfr_pipeline, because that pulls iclora_utils to utils.media_io to ltx_core.color.hlg, which imports PyAV — a codec binding this project deliberately does not vendor, and the same dependency the image_crf round trip is refused over. So the three helpers and three constants are lifted by AST from the real file and their OWN SOURCE TEXT is executed. Each name is asserted present, so an upstream rename raises rather than letting the generator quietly substitute a local definition for the thing under test. The generator also refuses a dirty checkout, so the recorded SHA always describes the code that ran.

The tripwire that fired, on purpose

The #920 refusal declared ABSENT HERE: GeneratedKeyframe, generated_keyframe, and test_ltx2_video re-derived those names from ltx2_conditioning.h. Section 4a of that spec predicted this exact event in terms:

If the readback lands, GeneratedKeyframe appears in the header, ABSENT goes red, and whoever landed it is told the refusal is now false.

It went red. The assertions are retired with the refusal they described and replaced by assertions about what replaced it, never widened. AGENTS.md forbids making a red gate green by deleting an assertion; this is the other case, where the subject of the assertion no longer exists.

Two upstream comments that overstate themselves

Both were found by mutations that stayed GREEN, and in both cases the green is a fact about the code rather than a hole in a test. Recorded because a later reader who removes either construct would otherwise have no way to tell.

• dfr_layout.py:177 gives the first tile no lead-in, which reads as load-bearing and is redundant with the max(0, own_lo - lead) clamp beside it. Measured against executed upstream: _build_tile on tile 0 with lead 0, 1 and 5 returns three TileRanges that compare EQUAL. ltx2_dfr.h carried the opposite claim until the mutation refuted it, and now carries the correction with a gate on it.
• keyframe_slots.py:161-163 says the causal fix "must not also be applied" because the span is set explicitly. Measured the same way: computing with causal_fix=True and then applying the explicit span gives a BYTE-IDENTICAL tensor, because the fix touches only the temporal axis and the span overwrites it in full.

Both are mirrored anyway, to follow upstream's text rather than its algebra.

Reachability

R1 deletes the production Ltx2DfrResolveCanvas call site and R2 deletes the production slot-conditioning call site. Both take test_ltx2_video RED, exit 1, having BUILT with zero compile errors and a CHANGED binary hash. A green there would have meant the DFR path is a test-only driver.

Two mutations stayed green against the engine suite, and the first explanation for one of them was wrong. Both are reported with what was measured, because a green mutation with a confident story attached is worse than one with none.

M8 needs an appending item AFTER the slot item, and upstream has exactly one: VideoConditionByReferenceLatent (dfr_pipeline.py:366-373), whose arm #975 owes. Until that lands the slots always trail and both readings agree. Gated in test_ltx2_dfr, which builds that configuration by hand and where M8 goes RED.

M5 was first explained as "a no-op on phase 0, which is the only phase the engine's DFR case runs" — an explanation that predicts a two-phase engine case would turn it RED. A two-phase case was then written, using the fixture spatial upsampler, rendering at full resolution and asserting stage 2 places and reads back its own slots at 4 tokens each against stage 1's 1. M5 stayed GREEN with it present, 52/52, exit 0. So the explanation was wrong.

The real reason is a property of the pipeline: stage 2 re-noises to stage_2_sigmas[0], about 0.909, so the seed is almost entirely replaced by noise before the first step and the loop generates the rest. The assertions an engine test can make about a slot are structural — counts, positions, resolutions — and the seed moves none of them; there is no A/B either, because the seed is internal rather than a request field. The two-phase case is kept for what it does establish, and its comment now says exactly that: it REACHES the seeded path, which no other engine test does, and it does not DETECT the seed. M5 stays gated in test_ltx2_dfr on the property that belongs to the port — the seed lands in latent and not in clean — where it goes RED.

Mutations

Every one carries three facts: git diff --stat after applying, whether it BUILT with the compile-error count beside it, and the exit code. Binary hashes were recorded before and after each rebuild, so no result is a stale binary reporting the previous run. The tree restored byte-identical and the binary returned to its baseline hash.

#	mutation	suite	result	errors	exit
M1	choose_segment_length tie takes the SMALLER candidate	test_ltx2_dfr	RED	0	1
M2	_build_tile keeps boundary 0 as an anchor	test_ltx2_dfr	RED	0	1
M3	drop_latent_prefix omits the seam handover +1	test_ltx2_dfr	RED, 18 failed	0	1
M4	NEGATIVE CONTROL: the first tile is given a lead-in	test_ltx2_dfr	GREEN (a no-op by algebra, proven upstream)	0	0
M4b	the lead-in is dropped from EVERY tile	test_ltx2_dfr	RED	0	1
M5	the slot seed never reaches latent	test_ltx2_dfr	RED	0	1
M5b	the slot seed ALSO goes into clean	test_ltx2_dfr	RED	0	1
M6	the slot append passes marked=false	test_ltx2_dfr	RED	0	1
M7	NEGATIVE CONTROL: the slot span applies the causal fix	test_ltx2_dfr	GREEN (byte-identical upstream)	0	0
M7b	the slot span is a full latent frame, not one pixel frame	test_ltx2_dfr	RED	0	1
M8	the extraction ASSUMES the slots trail	test_ltx2_dfr	RED	0	1
M8b	the layout records the wrong first_token	test_ltx2_dfr	RED	0	1
M11	the slot seed rounds half-away-from-zero, not half-to-even	test_ltx2_dfr	RED	0	1
M12	the carry-forward merge lets the ANCHOR win a shared position	test_ltx2_dfr	RED	0	1
M13	CONTROL: a no-op edit	test_ltx2_dfr	GREEN	0	0
R1	the production Ltx2DfrResolveCanvas call site is deleted	test_ltx2_video	RED	0	1
R2	the production slot-conditioning call site is deleted	test_ltx2_video	RED	0	1
M9	the slots are extracted AFTER the trim	test_ltx2_video	RED	0	1
M10	the DFR trim is dropped	test_ltx2_video	did NOT BUILD, 1 compile error — establishes nothing	1	n/a
M14	the canvas is resolved BEFORE assert_resolution (upstream's order reversed)	test_ltx2_video	RED	0	1
M15	the DFR audio cut is disabled	test_ltx2_video	RED, picture 0.375s, sound 1.01s, 2 channels at 48000 Hz	0	1

Gate

CPU only. The GPU was not used: a render ladder holds dgx.casa.

cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DVLLM_CPP_CUDA=OFF
cmake --build build -j6
ctest --test-dir build -j4 --output-on-failure

Re-run on the MERGED tree at head ad3a5e021; the last commit is documentation only. The final run is CTEST_EXIT=0, 491/491 passed, 167.17 s. origin/main moved three times during this row (0e1bee42f to 3005447f8 to e770a392f to 4496ef196) and each merge was followed by a full re-run, because a clean merge is not a building merge.

• CONFIGURE_EXIT=0, BUILD_EXIT=0
• ': error:' 0, ': warning:' 0, No space left 0, BFD assertion 0, against 487 Linking lines as the positive control that those greps ran over real content
• ctest -N = 491
• All 11 LTX-2.5 suites pass in every run, including the two this PR adds to: test_ltx2_dfr (11 cases / 652 assertions) and test_ltx2_video (52 cases / 1279 assertions)
• Skipped by their own guards: test_modelopt_mixed_precision_checkpoint, test_voxtral_e2e. Nothing else skipped
• 69-70 GB free disk throughout

The parallel runs were not all clean, and the reds are shown rather than dropped. One run returned CTEST_EXIT=0, 491/491 passed, at load average 6.93. Two others returned CTEST_EXIT=8 — test_engine_core_proc + test_cpu_threadpool in one, test_engine_core_proc + test_async_llm in the other, the latter at load average 58.55 on a box another session was hammering. Every one of those four passed when re-run ALONE, exit 0 each. All are on the standing load-dependent list (#294 and the threadpool ratio guard), and none is in this diff: git diff --name-only origin/main...HEAD | grep -iE 'threadpool|engine_core_proc|async_llm' is empty against 18 changed files as the control.

docs/FEATURES.md is a keyed record and both sides edited it. The automatic three-way merge is not accepted on its own word. AGENTS.md asks for the target version plus the scoped edit reapplied with unrelated keys proven byte-for-byte equal, and that was checked: 189 unrelated keys compare byte-for-byte equal, the new DFR row is present and is not a duplicate of anything on the target, and the LTX-2.5 DiT row differs from the target by exactly the six characters this row owns. The verifier carries its own floor — a comparison of fewer than 50 keys fails, because a parser that matched almost nothing would otherwise report a pass over an empty set.

• check-doc-checkpoint run per commit (#573): all seven commits exit 0. The instrument is ARMED, not merely quiet: --commit b5618b305 exits 1 with its known message, captured directly rather than through a pipe — which mattered, because the piped form reported exit 0 for a checker that had plainly just printed ERROR
• check-commit-style exit 0, check-commit-trailers exit 0, check-public-doc-tables exit 0, check-agent-record exit 0

One pre-existing red, filed rather than absorbed. scripts/agent-preflight.sh fails check-env-doc and test_check_env_doc on this branch and on every branch cut from 3005447f8: three VT_MOE_EXPERT_STREAM* knobs landed undocumented with #993. Proved pre-existing with a matched-arm check rather than asserted — the three sites are in qwen3_5.cpp, which this branch does not touch, and git diff origin/main...HEAD | grep '^+.*VT_MOE_EXPERT' returns nothing. Filed as #995, owner ENG-EXPERT-STREAM, with an index row in this branch.

Two defects this PR shipped and then found in itself

Both were found after the first push, by re-deriving this row's own upstream anchors at the final tree rather than trusting the ones written during implementation. All 44 citations resolve into their cited spans, 0 STALE — and twice the anchor was right while the CODE was not. The needle for each was written from the CLAIM the anchor is cited for rather than copied out of the span, because that check is circular and has reported 27/27 FRESH here while five anchors pointed at unrelated code.

The refusal order was reversed. The DFR canvas was resolved before Ltx2AssertResolution, where upstream calls assert_resolution at dfr_pipeline.py:291 and resolve_canvas at :314. A caller wrong on both the resolution and the frame count heard the wrong one of two correct refusals. Invisible to everything else, because on that input neither port renders anything. Gated by M14.

The soundtrack was not cut to the picture. dfr_pipeline.py:552-560 cuts decoded audio to num_frames / playback_fps, and upstream states the consequence rather than the mechanism: "Audio was generated for the padded canvas, so cut it to the video's duration or the muxed container outlasts the picture." The canvas pads 9 frames to 25; ashape.frames is derived from the PADDED count inside the phase loop, audio_lf carries it out, and the vocoder runs over all of it, so trimming the video latent touches none of it. A 9-frame DFR request emitted 0.375 s of picture beside 1.01 s of sound. Gated by M15.

The second is the more useful record, and the worse half is not the missing line. This PR shipped a comment saying the cut was "already implied by the trimmed frames", so a reader who went to check :552-560 would have found it accounted for and stopped. That comment is deleted and replaced by one that says what was wrong with it.

The first draft of the audio gate was itself wrong, in a way that reads as caution: it assumed one channel, calling that "the loosest reading and therefore the safe direction". It is the opposite — dividing by too few channels reports a LONGER file than exists, so the bound false-fails on a correct cut, which is exactly what it did. It now reads the channel count, sample rate and bit depth out of the WAV header rather than trusting the engine's own report of what it wrote.

What is owed, and why it is not here

The rounds loop needs the per-tile denoise pass as a callable. Upstream's rounds invoke the same DiffusionStage.__call__ the two stages use, per tile, with their own sigmas, stepper and seed (dfr_pipeline.py:480-499); this engine's denoise is inline inside one 680-line per-phase loop with no seam a tile can enter through. That is a refactor of the render path rather than an addition to it, so it is refused by name and listed under ## Owed against #986 rather than half-built. The refusal names the loop, names three ruled-out causes with what ruled each one out, and is gated by a case that re-derives its LOCAL claims from ltx2_dfr.h — because a refusal asserting only upstream symbol names cannot detect its own staleness, which this campaign has already paid for.

ltx-2.5-latent-temporal-upscaler-x2-bf16-1.0.safetensors is also still absent from the NAS, re-verified 2026-08-16 at the start and again at the end of this row, so a complete loop would have no real weights to run either.

Unverified, stated as unverified: nothing here has run on a real LTX-2.5 checkpoint. Every result is at reduced fixture dimensions on CPU, and no real-weight DFR render exists.

Arms

Arm	State
bf16 / f32 reference	served
FP8	served — the DFR path and the slot request are resolved ahead of arm selection, so no arm reaches the machinery by a different route
NVFP4	served, same reason
GGUF k-quants	not applicable. LTX-2.5 ships no GGUF: the four quantization kinds upstream defines are fp8-cast, fp8-scaled-mm, nvfp4-cast and nvfp4-prequant (quantization_factory.py:23-26), and no published LTX-2.5 checkpoint is a GGUF
temporal rounds	refused by name, owed by #986
stage-2 detailing IC-LoRA	refused by name, owed by #975

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [claude-code]