[TRTLLM-11319][feat] VisualGen public output API + bench timing decomposition

[zhenhuaw-me]

Description

Reworks the public output story of tensorrt_llm.visual_gen so callers get a coherent, request-aware return type with measurable engine-side timing — and reworks the offline bench so its latency and engine generation metrics line up with what the serving path does.

Public output API

• VisualGenOutput replaces the raw MediaOutput as the user-facing return type. Carries image/video/audio tensors plus request_id, error, frame_rate / audio_sample_rate, and engine-side VisualGenMetrics.
• VisualGenResult is a single Future-like awaitable handle that resolves to VisualGenOutput (single prompt) or List[VisualGenOutput] (batch). Batched generate(List[str]) fans out into per-item outputs with per-item error semantics.
• VisualGenOutput.save(path) is the single user-facing way to persist a generated tensor to disk (image PNG/JPG/WEBP, video MP4/AVI).
• Public exceptions: dropped VisualGenError / VisualGenParamsError in favor of standard RuntimeError / ValueError / NotImplementedError. Aligns with vLLM/SGLang convention where diffusion engines raise standard exception types and callers catch by built-in type, not by package-specific wrappers.

Internal types and contracts

• Internal MediaOutput renamed to PipelineOutput and gains three CUDA-event timing phases (pre_denoise / denoise / post_denoise) plus an executor-measured generation field on the wire response.
• Tightened batched-shape contract: image is always (B, H, W, C), video (B, T, H, W, C), audio (B, channels, T_audio). Asserts in split_visual_gen_output make a violating pipeline fail loudly rather than silently corrupt per-item outputs by indexing along the wrong dim. LTX-2's decode_audio no longer unconditionally .squeeze(0)s, restoring the contract for B=1.

Module boundaries

• Tensor-encoding code extracted from tensorrt_llm/serve/media_storage.py into a new tensorrt_llm/media/encoding.py module of free functions; MediaStorage keeps its file-storage role under serve/.
• The eight video-generation route handlers in openai_server.py are extracted into a sibling mixin (tensorrt_llm/serve/openai_video_routes.py) so openai_server.py stays under the per-file line budget. Behavior is strictly preserved.

Engine reliability

• Fixed a timeout-leak in DiffusionRemoteClient: previously, a request that timed out in aresult() could have its late-arriving response pinned in completed_responses for the process lifetime, leaking a full PipelineOutput (including video tensors) per timeout. New abandon_request_id flow uses an abandoned-id set so _store_response drops late responses and abandon_request_id pops cached ones; both orderings handled under the same async lock.

Server endpoints

• /v1/images/edits short-circuits to HTTP 501 NotImplementedError. Reason: no in-tree pipeline implements image editing today (Flux/Flux2 are gen-only; Wan/LTX-2 produce video). The handler used to dispatch through the generator and 500 on a downstream None check; 501 is the honest answer until an edit-capable pipeline lands. Route stays registered so re-enabling is a one-line change.

Benchmark (tensorrt_llm/bench/benchmark/visual_gen.py)

• Per-run persisted media dir: _run_benchmark wraps each run in one tempfile.TemporaryDirectory and saves each result into it. Mirrors the server's persisted-write pattern (no per-request mkstemp / unlink overhead in the timing window).
• ffmpeg-optional fallback via resolve_video_format("auto"), matching how the server treats output_format="auto". The bench no longer hard-fails on hosts without ffmpeg.
• Timing units: dropped _ms suffix and store wall-clock seconds end-to-end. CudaPhaseTimer.fill() divides Event.elapsed_time by 1000 once at the boundary so every downstream type carries seconds.
• Renamed the engine-side metric from pipeline (a class-name leak) to generation (describes what the timer measures, not how). Threaded through VisualGenMetrics, DiffusionResponse, server log lines, bench per-request and aggregate types.
• VisualGenBenchmarkMetrics gains mean / median / std / min / max / percentiles_generation alongside the existing latency aggregates. The latency >= generation gap is the encode + persist + IPC overhead the bench measures around the engine, and the report exposes that headroom.

API stability

YAML coverage for the VisualGen public surface is explicitly deferred until the API exits prototype status (all classes are @set_api_status("prototype")).

Test Coverage

• tests/unittest/visual_gen/test_output.py — VisualGenOutput dataclass, VisualGenMetrics, to_visual_gen_output / split_visual_gen_output success and error paths, batch fan-out, VisualGenOutput.save() routing, VisualGenResult sync result(), async aresult(), __await__, timeout-leak fix (5 new tests covering abandon_request_id's two race orderings via a minimal DiffusionRemoteClient stub).
• tests/unittest/_torch/visual_gen/test_visual_gen_params.py — request-validation surface migrated to plain ValueError.
• tests/unittest/_torch/visual_gen/test_trtllm_serve_endpoints.py — FastAPI mock tests; TestImageEdit reduced to a 501 assertion plus the existing pydantic-validation 400 test.
• tests/unittest/media/test_encoding.py — PNG round-trip, video_to_bytes AVI bytes, resolve_video_format dispatch, ffmpeg fallback paths.
• tests/integration/defs/examples/test_visual_gen.py — Wan T2V and Flux end-to-end public-output contract checks (single + batch + async), including error-path semantics. Field-name assertions updated for the timing rename.
• tests/integration/defs/visual_gen/test_visual_gen_benchmark.py — bench JSON-key assertions extended with mean_generation.

Summary by CodeRabbit

• New Features

  • Added performance metrics (pipeline latency, denoising timing) to generation outputs via VisualGenMetrics
  • Introduced VisualGenOutput.save() method for easy output persistence
  • Added OpenAI-compatible video generation endpoints

• Improvements

  • Refactored output handling for cleaner result objects with error tracking
  • Consolidated media encoding functionality into dedicated utilities

• Documentation

  • Updated all visual generation examples to use new output API

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.

• PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions)

• Any new dependencies have been scanned for license and vulnerabilities

• CODEOWNERS updated if ownership changes

• Documentation updated as needed

• Update tava architecture diagram if there is a significant design change in PR.

• The reviewers assigned automatically/manually are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

[zhenhuaw-me]

/bot run --disable-fail-fast

[coderabbitai]

📝 Walkthrough

Walkthrough

The changes refactor the visual generation output architecture, replacing MediaOutput with a new internal PipelineOutput type enhanced with CUDA timing instrumentation via CudaPhaseTimer, introducing public VisualGenOutput and VisualGenMetrics types as the primary user-facing API, migrating media encoding logic from MediaStorage into a new dedicated tensorrt_llm.media.encoding module, updating all example scripts to use output.save(), extracting OpenAI video routes into a reusable _VideoRoutesMixin, and extending test coverage for the new output and encoding APIs.

Changes

Cohort / File(s)	Summary
Example Output Persistence examples/visual_gen/models/wan_t2v.py, examples/visual_gen/quickstart_example.py, examples/visual_gen/visual_gen_flux.py, examples/visual_gen/visual_gen_ltx2.py, examples/visual_gen/visual_gen_wan_i2v.py, examples/visual_gen/visual_gen_wan_t2v.py	Removed MediaStorage imports and replaced MediaStorage.save_*() calls with output.save(args.output_path) for all generated visual content (images and videos).
Public Output API tensorrt_llm/visual_gen/output.py	Added VisualGenMetrics and VisualGenOutput dataclasses as public result containers with timing metadata and a save() method for persisting generated media. Replaced MediaOutput with these new types.
Internal Pipeline Output tensorrt_llm/_torch/visual_gen/output.py	Introduced PipelineOutput internal dataclass replacing MediaOutput with frame rate and audio sample rate metadata. Added CudaPhaseTimer helper for measuring CUDA-event-based phase timings (pre-denoise, denoise, post-denoise). Added conversion utilities to_visual_gen_output() and split_visual_gen_output() for transforming responses into public output types.
Internal Output Exports tensorrt_llm/_torch/visual_gen/__init__.py, tensorrt_llm/visual_gen/__init__.py	Updated module exports: replaced MediaOutput with PipelineOutput and VisualGenMetrics/VisualGenOutput re-exports in __all__ lists.
Main Package Exports tensorrt_llm/__init__.py	Added VisualGenMetrics and VisualGenOutput to root-level exports for public API access.
Pipeline Implementations tensorrt_llm/_torch/visual_gen/models/flux/pipeline_flux.py, tensorrt_llm/_torch/visual_gen/models/flux/pipeline_flux2.py, tensorrt_llm/_torch/visual_gen/models/ltx2/pipeline_ltx2.py, tensorrt_llm/_torch/visual_gen/models/ltx2/pipeline_ltx2_two_stages.py, tensorrt_llm/_torch/visual_gen/models/wan/pipeline_wan.py, tensorrt_llm/_torch/visual_gen/models/wan/pipeline_wan_i2v.py	Replaced MediaOutput return type with PipelineOutput wrapped in CudaPhaseTimer. Added CUDA phase markers (mark_pre_start, mark_denoise_start, mark_post_start, mark_end) around denoising and decoding stages to instrument per-phase latencies. Updated docstrings to reflect PipelineOutput.
Executor Response Type tensorrt_llm/_torch/visual_gen/executor.py	Updated DiffusionResponse dataclass: changed output field from MediaOutput to PipelineOutput, added pipeline_ms field. Instrumented request processing with host-side wall-clock timing around pipeline.infer() using time.perf_counter().
Main VisualGen API tensorrt_llm/visual_gen/visual_gen.py	Updated VisualGenResult to be awaitable and resolve to VisualGenOutput or List[VisualGenOutput]. Changed VisualGen.generate() return type from MediaOutput to Union[VisualGenOutput, List[VisualGenOutput]]. Added input validation and batch-size tracking. Updated error semantics: single prompts raise on error, batch items return errors within output objects.
Media Encoding Module tensorrt_llm/media/__init__.py, tensorrt_llm/media/encoding.py	Created new tensorrt_llm.media.encoding module with public tensor-based encoding functions: save_image(), image_to_bytes(), save_video(), video_to_bytes(). Supports both ffmpeg-based MP4 encoding and pure-Python MJPEG/AVI fallback. Includes optional audio muxing and format resolution logic.
MediaStorage Reduction tensorrt_llm/serve/media_storage.py	Removed all media encoding/persistence methods (save_image, convert_image_to_bytes, save_video, convert_video_to_bytes, internal helpers, and ffmpeg handling). Left only documentation redirecting to new tensorrt_llm.media.encoding module.
OpenAI Server Video Routes tensorrt_llm/serve/openai_server.py, tensorrt_llm/serve/openai_video_routes.py	Created _VideoRoutesMixin providing eight OpenAI-compatible video endpoints (sync/async generation, job listing/retrieval/deletion, content serving, metadata access). Refactored OpenAIServer to inherit from mixin. Switched image encoding from MediaStorage to tensorrt_llm.media.encoding. Added end-to-end timing and metrics extraction from VisualGenOutput.metrics.
Benchmark Metrics tensorrt_llm/bench/benchmark/visual_gen.py, tensorrt_llm/bench/benchmark/visual_gen_utils.py	Added _save_to_tempfile() helper to include encoding latency in end-to-end measurements. Updated VisualGenRequestOutput with pipeline_ms and denoise_ms fields. Refactored result capture to extract metrics from result.metrics.
Test Files tests/integration/defs/examples/test_visual_gen.py, tests/unittest/_torch/visual_gen/test_media_storage.py, tests/unittest/_torch/visual_gen/test_trtllm_serve_endpoints.py, tests/unittest/dynamo/test_imports.py, tests/unittest/media/test_encoding.py, tests/unittest/visual_gen/test_output.py	Deleted obsolete MediaStorage unit tests. Added comprehensive new tests for VisualGenOutput, VisualGenMetrics, PipelineOutput, media encoding functions, and VisualGenResult awaitability. Updated endpoint tests to mock VisualGenOutput instead of MediaOutput. Updated import validation test to expect PipelineOutput export. Updated integration tests to use output.save() and validate output metadata.

Sequence Diagram(s)

sequenceDiagram
    participant Client
    participant VisualGen
    participant Executor
    participant Pipeline
    participant CudaPhaseTimer
    participant Encoder

    Client->>VisualGen: generate(inputs)
    VisualGen->>Executor: submit request (prompt, params)
    Executor->>Pipeline: create CudaPhaseTimer
    Pipeline->>CudaPhaseTimer: mark_pre_start()
    Pipeline->>Pipeline: initialization
    Pipeline->>CudaPhaseTimer: mark_denoise_start()
    Pipeline->>Pipeline: denoising loop
    Pipeline->>CudaPhaseTimer: mark_post_start()
    Pipeline->>Pipeline: decode video/image
    Pipeline->>CudaPhaseTimer: mark_end()
    CudaPhaseTimer->>Pipeline: fill(PipelineOutput)
    Pipeline->>Executor: return PipelineOutput with metrics
    Executor->>Executor: measure pipeline_ms (host-side)
    Executor->>VisualGen: DiffusionResponse(output=PipelineOutput, pipeline_ms)
    VisualGen->>VisualGen: convert to VisualGenOutput
    VisualGen->>Client: return VisualGenOutput
    Client->>Encoder: output.save(path)
    Encoder->>Encoder: encode image/video
    Encoder->>Encoder: write to file
    Encoder->>Client: return path

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 66.03% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title accurately summarizes the main change: introducing a public VisualGenOutput API and benchmark timing decomposition for the visual generation feature.
Description check	✅ Passed	PR description is comprehensive with clear sections on public API changes, internal types, module boundaries, engine reliability improvements, server endpoint changes, and benchmark updates.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

Tip

💬 Introducing Slack Agent: The best way for teams to turn conversations into code.

Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.

• Generate code and open pull requests
• Plan features and break down work
• Investigate incidents and troubleshoot customer tickets together
• Automate recurring tasks and respond to alerts with triggers
• Summarize progress and report instantly

Built for teams:

• Shared memory across your entire org—no repeating context
• Per-thread sandboxes to safely plan and execute work
• Governance built-in—scoped access, auditability, and budget controls

One agent for your entire SDLC. Right inside Slack.

👉 Get started

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 7

🧹 Nitpick comments (3)

tests/integration/defs/examples/test_visual_gen.py (1)

482-490: ⚡ Quick win

Assert audio_sample_rate on the two-stage LTX-2 path too.

The single-stage path checks that rate propagation worked, but this one doesn't. If two-stage output stops populating audio_sample_rate, output.save(...) can still pass by falling back to its default, so this regression would go uncaught.

Suggested test addition

         assert output.error is None
         assert output.video is not None
         assert output.frame_rate == LTX2_T2V_FRAME_RATE
+        assert output.audio_sample_rate is not None and output.audio_sample_rate > 0
         assert output.metrics is not None
         assert output.metrics.pipeline_ms > 0
         assert output.metrics.denoise_ms > 0

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@tests/integration/defs/examples/test_visual_gen.py` around lines 482 - 490,
Add an assertion that the two-stage LTX-2 output populates audio_sample_rate:
after calling visual_gen.generate(...) and before output.save(...), assert that
output.audio_sample_rate equals the expected constant (e.g.
LTX2_T2V_AUDIO_SAMPLE_RATE) so that regressions where audio_sample_rate is
missing are caught; locate this in the same block that calls visual_gen.generate
and uses output.save to ensure parity with the single-stage test.

tests/unittest/media/test_encoding.py (2)

94-116: ⚡ Quick win

Make the AVI fallback tests deterministic.

These cases currently depend on the host having no ffmpeg. On builders where ffmpeg is present, save_video(..., format="avi") can take the ffmpeg path instead, so the test stops validating the pure-Python fallback it documents. Patch ffmpeg availability to False and reset the cached encoder in the test to pin the intended branch.

As per coding guidelines, "Coverage expectations: Assess whether new/changed tests cover happy path, important edge cases, and failure modes relevant to the feature or fix."

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@tests/unittest/media/test_encoding.py` around lines 94 - 116, Tests that
exercise the pure-Python AVI fallback must force the non-ffmpeg branch; in the
two tests calling video_to_bytes and save_video (symbols: video_to_bytes,
save_video, _dummy_video) use monkeypatch to set whatever ffmpeg-availability
flag/function your video module exposes to False (e.g., patch a function like
ffmpeg_available() to return False or set a module-level FFMPEG_AVAILABLE =
False), then clear/reset the module encoder cache before calling the functions
(remove or reinitialize the cached encoder object/attribute used by the
pure-Python path so the fallback is chosen, e.g., delattr(video_module,
"<cached_encoder_name>") or call the provided reset function) so the tests
deterministically exercise the pure-Python AVI encoder.

---

1-24: QA list updates look unnecessary here.

These additions stay in tests/unittest/, so I don't see a corresponding need for changes under tests/integration/test_lists/qa/ in this PR.

As per coding guidelines, "If the PR only touches unittest/ or narrow unit scope, say explicitly whether QA list updates are unnecessary or optional."

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@tests/unittest/media/test_encoding.py` around lines 1 - 24, This PR only
modifies the unit test for the module tensorrt_llm.media.encoding (the test that
exercises image_to_bytes/save_image/video_to_bytes/resolve_video_format), so
explicitly state in the PR description or a commit message that QA list updates
under the integration QA lists are unnecessary for this change; update the PR
text to say "QA list updates unnecessary — change limited to unit tests" (or
similar), and remove or revert any unrelated changes made to the integration QA
list if present.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@examples/visual_gen/visual_gen_ltx2.py`:
- Line 416: The CLI help text for the --output_path argument is out of sync with
the actual save call (output.save(args.output_path)); find the
parser.add_argument call that defines '--output_path' and update its help string
to accurately list the formats output.save supports (replace the old ".gif/.png
video outputs" text with the correct supported file types/containers), so the
help matches the behavior of output.save and args.output_path.

In `@tensorrt_llm/_torch/visual_gen/output.py`:
- Around line 148-149: The current truthiness check on resp.error_msg allows
empty-string errors to be treated as success and lead to dereferencing
resp.output; change both checks (the one around VisualGenOutput creation and the
other at lines handling resp.output) to explicitly test for None (e.g., if
resp.error_msg is not None) so any non-None error_msg — including empty string —
short-circuits and returns VisualGenOutput(request_id=resp.request_id,
error=resp.error_msg) instead of accessing resp.output.

In `@tensorrt_llm/bench/benchmark/visual_gen_utils.py`:
- Around line 36-47: The new timing fields pipeline_ms and denoise_ms on the
visual generation result are not being propagated into the JSON/console outputs;
update calculate_metrics to aggregate these per-request fields (e.g.,
sum/count/min/max/percentiles as appropriate) alongside e2e_latency and then
modify build_visual_gen_result_dict to include the aggregated pipeline_ms and
denoise_ms values in the returned dict and any per-request entries; locate uses
of VisualGenOutput.metrics and the result dataclass fields (pipeline_ms,
denoise_ms, success, e2e_latency) to read the engine-side timings and ensure
they are serialized into the final metrics JSON/console output.

In `@tensorrt_llm/serve/openai_video_routes.py`:
- Around line 98-103: The except blocks currently map all non-ValueError
exceptions to the default (client) error response; update each generic except
Exception handler (the ones that log traceback and call
create_error_response(str(e))) to return an internal server error (500) instead
of the default 400 by invoking create_error_response with an explicit 500 status
(or using an existing create_internal_error_response helper if present).
Concretely, in the handlers around the create_error_response usages (e.g.,
inside the functions that catch ValueError and Exception, and at the other noted
sites) keep the ValueError branch as-is but change the generic Exception branch
to logger.error(traceback.format_exc()) followed by return
self.create_error_response(str(e), status=500) (or equivalent API) so
server-side failures are classified as InternalServerError. Ensure you apply the
same change in the other occurrences mentioned (the other except Exception
blocks).
- Around line 196-202: video_gen_tasks is never cleaned up and delete_video()
does not cancel running jobs; update the flow so tasks are removed and cancelled
when finished or deleted: when creating the task for _generate_video_background,
attach an add_done_callback that removes the entry from self.video_gen_tasks
(and logs exceptions from task.result()); modify delete_video() to look up the
task in self.video_gen_tasks, if present cancel it (task.cancel()), await or
suppress asyncio.CancelledError as appropriate, and then delete the key so a
deleted job cannot keep running or leak memory; apply the same pattern to other
places that spawn background generation tasks referenced in the file.
- Around line 195-217: Persist the VideoJob to VIDEO_STORE before scheduling the
background task: construct the VideoJob (VideoJob(...)) and call await
VIDEO_STORE.upsert(video_id, video_job) first, then create and assign the
background task to self.video_gen_tasks[video_id] which calls
self._generate_video_background(...); this ensures _generate_video_background
can always find the job by id (video_id) even if it completes quickly. Return
the JSONResponse after the upsert and task scheduling.

In `@tensorrt_llm/visual_gen/visual_gen.py`:
- Around line 599-609: The single-request timeout path currently sets
self._finished before populating self._resolved, so the first raise
VisualGenError is not reflected for subsequent aresult()/result() calls; change
both timeout branches (the block handling response is None and the similar block
at the other location) to first set self._resolved =
[VisualGenOutput(request_id=self.request_id, error="Generation timed out")] (or
a single VisualGenOutput for non-batch), then set self._finished = True, and
finally raise VisualGenError("Generation timed out") so that later calls return
the stored error state; update references in the response handling logic that
use self._batch_size, self._resolved, self._finished, VisualGenOutput, and
VisualGenError.

---

Nitpick comments:
In `@tests/integration/defs/examples/test_visual_gen.py`:
- Around line 482-490: Add an assertion that the two-stage LTX-2 output
populates audio_sample_rate: after calling visual_gen.generate(...) and before
output.save(...), assert that output.audio_sample_rate equals the expected
constant (e.g. LTX2_T2V_AUDIO_SAMPLE_RATE) so that regressions where
audio_sample_rate is missing are caught; locate this in the same block that
calls visual_gen.generate and uses output.save to ensure parity with the
single-stage test.

In `@tests/unittest/media/test_encoding.py`:
- Around line 94-116: Tests that exercise the pure-Python AVI fallback must
force the non-ffmpeg branch; in the two tests calling video_to_bytes and
save_video (symbols: video_to_bytes, save_video, _dummy_video) use monkeypatch
to set whatever ffmpeg-availability flag/function your video module exposes to
False (e.g., patch a function like ffmpeg_available() to return False or set a
module-level FFMPEG_AVAILABLE = False), then clear/reset the module encoder
cache before calling the functions (remove or reinitialize the cached encoder
object/attribute used by the pure-Python path so the fallback is chosen, e.g.,
delattr(video_module, "<cached_encoder_name>") or call the provided reset
function) so the tests deterministically exercise the pure-Python AVI encoder.
- Around line 1-24: This PR only modifies the unit test for the module
tensorrt_llm.media.encoding (the test that exercises
image_to_bytes/save_image/video_to_bytes/resolve_video_format), so explicitly
state in the PR description or a commit message that QA list updates under the
integration QA lists are unnecessary for this change; update the PR text to say
"QA list updates unnecessary — change limited to unit tests" (or similar), and
remove or revert any unrelated changes made to the integration QA list if
present.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 34454c9b-615c-41d0-8b29-1ba1656ba7fb

📥 Commits

Reviewing files that changed from the base of the PR and between 37fc0e3 and 13a9b96.

📒 Files selected for processing (34)

• examples/visual_gen/models/wan_t2v.py
• examples/visual_gen/quickstart_example.py
• examples/visual_gen/visual_gen_flux.py
• examples/visual_gen/visual_gen_ltx2.py
• examples/visual_gen/visual_gen_wan_i2v.py
• examples/visual_gen/visual_gen_wan_t2v.py
• tensorrt_llm/__init__.py
• tensorrt_llm/_torch/visual_gen/__init__.py
• tensorrt_llm/_torch/visual_gen/executor.py
• tensorrt_llm/_torch/visual_gen/models/flux/pipeline_flux.py
• tensorrt_llm/_torch/visual_gen/models/flux/pipeline_flux2.py
• tensorrt_llm/_torch/visual_gen/models/ltx2/pipeline_ltx2.py
• tensorrt_llm/_torch/visual_gen/models/ltx2/pipeline_ltx2_two_stages.py
• tensorrt_llm/_torch/visual_gen/models/wan/pipeline_wan.py
• tensorrt_llm/_torch/visual_gen/models/wan/pipeline_wan_i2v.py
• tensorrt_llm/_torch/visual_gen/output.py
• tensorrt_llm/bench/benchmark/visual_gen.py
• tensorrt_llm/bench/benchmark/visual_gen_utils.py
• tensorrt_llm/media/__init__.py
• tensorrt_llm/media/encoding.py
• tensorrt_llm/serve/media_storage.py
• tensorrt_llm/serve/openai_server.py
• tensorrt_llm/serve/openai_video_routes.py
• tensorrt_llm/visual_gen/__init__.py
• tensorrt_llm/visual_gen/output.py
• tensorrt_llm/visual_gen/visual_gen.py
• tests/integration/defs/examples/test_visual_gen.py
• tests/unittest/_torch/visual_gen/test_media_storage.py
• tests/unittest/_torch/visual_gen/test_trtllm_serve_endpoints.py
• tests/unittest/dynamo/test_imports.py
• tests/unittest/media/__init__.py
• tests/unittest/media/test_encoding.py
• tests/unittest/visual_gen/__init__.py
• tests/unittest/visual_gen/test_output.py

💤 Files with no reviewable changes (1)

• tests/unittest/_torch/visual_gen/test_media_storage.py

[zhenhuaw-me]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #46326 [ run ] triggered by Bot. Commit: a3cf840 Link to invocation

[tensorrt-cicd]

PR_Github #46326 [ run ] completed with state SUCCESS. Commit: a3cf840
/LLM/main/L0_MergeRequest_PR pipeline #36422 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[tensorrt-cicd]

PR_Github #46925 [ run ] completed with state SUCCESS. Commit: 45f1b61
/LLM/main/L0_MergeRequest_PR pipeline #36931 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[zhenhuaw-me]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #47085 [ run ] triggered by Bot. Commit: e148156 Link to invocation

[QiJune]

LGTM

[tensorrt-cicd]

PR_Github #47085 [ run ] completed with state SUCCESS. Commit: e148156
/LLM/main/L0_MergeRequest_PR pipeline #37056 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[zhenhuaw-me]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #47218 [ run ] triggered by Bot. Commit: 2499db9 Link to invocation

[brb-nv]

LGTM.

[tensorrt-cicd]

PR_Github #47218 [ run ] completed with state SUCCESS. Commit: 2499db9
/LLM/main/L0_MergeRequest_PR pipeline #37174 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

• Please check the failed tests and fix your PR
• If you cannot view the failures, ask the CI triggerer to share details
• Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

[zhenhuaw-me]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #47311 [ run ] triggered by Bot. Commit: 2499db9 Link to invocation

[tensorrt-cicd]

PR_Github #47311 [ run ] completed with state SUCCESS. Commit: 2499db9
/LLM/main/L0_MergeRequest_PR pipeline #37252 completed with status: 'SUCCESS'

CI Report

Link to invocation