Skip to content

[TRTLLM-13028][doc] Add VisualGen API walkthrough example and docs page#14685

Merged
zhenhuaw-me merged 1 commit into
NVIDIA:mainfrom
zhenhuaw-me:api-example
Jun 2, 2026
Merged

[TRTLLM-13028][doc] Add VisualGen API walkthrough example and docs page#14685
zhenhuaw-me merged 1 commit into
NVIDIA:mainfrom
zhenhuaw-me:api-example

Conversation

@zhenhuaw-me
Copy link
Copy Markdown
Member

@zhenhuaw-me zhenhuaw-me commented May 28, 2026

Description

Adds a slim, LLM-API-style end-to-end walkthrough of the public VisualGen Python API as examples/visual_gen/api_walkthrough.py, and wires up a "VisualGen Examples" page in the rendered docs that mirrors the LLM Examples page. The example walks through VisualGen.supported_models(), VisualGen.pipeline_config(model), VisualGenArgs(pipeline_config=..., compilation_config=...) with warmup skipped, visual_gen.extra_param_specs, visual_gen.default_params with a num_frames override (plus the matching extra_params override loop — no-op on Wan 2.1, real on Wan 2.2 A14B), and VisualGenOutput.save.

The docs piece scans examples/visual_gen/*.py at Sphinx build time through docs/source/helper.py (same generator that builds the LLM API and trtllm-serve examples pages), produces docs/source/examples/visual_gen_examples.rst, and mounts it between examples/llm_api_examples.rst and examples/trtllm_serve_examples in the Deployment Guide toctree. Existing visual_gen_*.py / .sh CLI scripts without ### :title metadata are temporarily added to helper.py's ignore_list; they can opt in later by adding the metadata block.

The doc structure and the rendered version: https://sc.talos.nvidia.com/view/home/scratch.zhenhuaw_sw/llm/aigv/docs/build/html/index.html

截屏2026-05-28 下午5 37 43

Test Coverage

  • New integration test test_visual_gen_api_walkthrough registered in the same L0 lists that already exercise test_visual_gen_quickstart: l0_l40s.yml, l0_gh200.yml, l0_h100.yml, l0_dgx_b200.yml.
  • End-to-end verified locally on a GB200: the example boots VisualGen with warmup skipped, prints all four introspection sections, generates, and saves a valid 832x480 / 101-frame / 16 fps H.264 AVI to disk.
  • Sphinx make html builds clean with no new warnings on api_walkthrough.rst / visual_gen_examples.rst; the "VisualGen Examples" entry shows up between LLM Examples and Online Serving Examples in the side nav.

PR Checklist

Please review the following before submitting your PR:

  • PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
  • PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
  • Test cases are provided for new code paths (see test instructions)
  • If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.
  • Any new dependencies have been scanned for license and vulnerabilities
  • CODEOWNERS updated if ownership changes
  • Documentation updated as needed
  • Update tava architecture diagram if there is a significant design change in PR.
  • The reviewers assigned automatically/manually are appropriate for the PR.
  • Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

Summary by CodeRabbit

  • New Features

    • Added a VisualGen API example demonstrating end-to-end usage including model selection, pipeline configuration, parameter customization, and video generation with output saving.
  • Documentation

    • Expanded documentation to include VisualGen example resources and updated the main documentation index.
  • Tests

    • Added integration test for VisualGen API walkthrough across multiple GPU configurations.

Review Change Stack

Add examples/visual_gen/api_walkthrough.py — a slim, LLM-API-style
walkthrough of the public VisualGen API on Wan 2.1 T2V that covers, in
order:

- VisualGen.supported_models() to list registered pipelines
- VisualGen.pipeline_config(model) for per-architecture knobs
- VisualGenArgs(pipeline_config=..., compilation_config=...) with the
  post-load warmup skipped
- visual_gen.extra_param_specs to introspect model-specific
  extra_params keys
- visual_gen.default_params with a num_frames override (and the
  matching extra_params override loop, no-op on Wan 2.1)
- VisualGenOutput.save to persist the .avi to disk

Add a "VisualGen Examples" section to the rendered TRT-LLM docs that
mirrors the LLM Examples page, generated from examples/visual_gen/*.py
at Sphinx build time via docs/source/helper.py. The new index sits
between LLM Examples and Online Serving Examples in the Deployment
Guide toctree.

Register test_visual_gen_api_walkthrough in the same L0 lists that
already exercise test_visual_gen_quickstart (l0_l40s.yml,
l0_gh200.yml, l0_h100.yml, l0_dgx_b200.yml).

Signed-off-by: Zhenhua Wang <zhenhuaw@nvidia.com>
@zhenhuaw-me zhenhuaw-me added api-compatible Accepted LLM API contract change that is backwards-compatible Doc <NV>TRTLLM's textual/illustrative materials: API refs, guides, tutorials. Improvement & clarity. VisualGen labels May 28, 2026
@zhenhuaw-me zhenhuaw-me marked this pull request as ready for review May 28, 2026 10:01
@zhenhuaw-me zhenhuaw-me requested review from a team as code owners May 28, 2026 10:01
@zhenhuaw-me
Copy link
Copy Markdown
Member Author

/bot run

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 28, 2026

📝 Walkthrough

Walkthrough

This PR adds a VisualGen API example script with full documentation and integration testing. The changes update documentation generation to produce VisualGen example docs, integrate them into the main index, implement the example walkthrough, add an integration test to validate it, and register the test across multiple GPU test matrices.

Changes

VisualGen API Walkthrough Example

Layer / File(s) Summary
Documentation generation for VisualGen examples
docs/source/helper.py
Updated generate_examples() to collect scripts from examples/visual_gen, expanded the ignore list with VisualGen-specific entries and comments, and generate a dedicated visual_gen_examples.rst toctree.
Documentation index integration
docs/source/index.rst
Added the generated examples/visual_gen_examples.rst to the "Deployment Guide" toctree.
VisualGen API walkthrough example script
examples/visual_gen/api_walkthrough.py
New example demonstrating end-to-end VisualGen API usage: lists supported models, retrieves pipeline config, constructs VisualGen instance with compilation config, inspects extra param specs, builds and overrides request parameters, generates media output, and saves to AVI.
Integration test for API walkthrough example
tests/integration/defs/examples/test_visual_gen.py
Added test_visual_gen_api_walkthrough that verifies the example runs successfully by checking for the Wan 2.1 T2V model, symlinking the checkpoint into venv workspace, executing the example script, and asserting the .avi output is produced.
Test matrix registration across GPU configurations
tests/integration/test_lists/test-db/l0_dgx_b200.yml, l0_gh200.yml, l0_h100.yml, l0_l40s.yml
Registered test_visual_gen_api_walkthrough in four GPU-specific test matrices under their respective condition blocks.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Suggested reviewers

  • QiJune
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 66.67% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and specifically summarizes the main changes: adding a VisualGen API walkthrough example and corresponding documentation page.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description check ✅ Passed The PR description comprehensively explains the changes, including the new API walkthrough example, documentation integration, test coverage, and verification details.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
tests/integration/defs/examples/test_visual_gen.py (1)

1177-1197: Coverage is sufficient for this new example path.

tests/integration/defs/examples/test_visual_gen.py::test_visual_gen_api_walkthrough validates execution + output artifact, and the PR also wires it into tests/integration/test_lists/test-db/l0_dgx_b200.yml, tests/integration/test_lists/test-db/l0_gh200.yml, tests/integration/test_lists/test-db/l0_h100.yml, and tests/integration/test_lists/test-db/l0_l40s.yml.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/integration/defs/examples/test_visual_gen.py` around lines 1177 - 1197,
The test test_visual_gen_api_walkthrough manipulates model_dst by creating
directories and a symlink but doesn't guard against an existing non-symlink
destination nor perform cleanup, which can cause flaky runs; update the test to
(1) if model_dst exists and is not a symlink, fail or skip with a clear message,
(2) create the parent dirs and symlink only when model_dst is absent, and (3)
remove the created symlink (and any empty parent dir you created) in a
finally/teardown block after venv_check_call finishes; reference model_src,
model_dst, venv_check_call and output_path when making these changes.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@examples/visual_gen/api_walkthrough.py`:
- Line 7: The function main is missing an explicit return type annotation;
update the definition of main in api_walkthrough.py to include a return type
(e.g., def main() -> None:) since this module requires all functions be
annotated — ensure the signature uses -> None if it does not return a value and
adjust any callers if you change the contract.
- Around line 1-4: This new module is missing the required NVIDIA SPDX
copyright/license header; add the standard NVIDIA SPDX header block at the very
top of the file before any metadata or imports so the file begins with the
license text followed by the existing lines that import VisualGen,
VisualGenArgs, and CompilationConfig; ensure the exact company and SPDX
identifier used across the repo is used verbatim and that the header precedes
the "### :title" metadata and the imports.

---

Nitpick comments:
In `@tests/integration/defs/examples/test_visual_gen.py`:
- Around line 1177-1197: The test test_visual_gen_api_walkthrough manipulates
model_dst by creating directories and a symlink but doesn't guard against an
existing non-symlink destination nor perform cleanup, which can cause flaky
runs; update the test to (1) if model_dst exists and is not a symlink, fail or
skip with a clear message, (2) create the parent dirs and symlink only when
model_dst is absent, and (3) remove the created symlink (and any empty parent
dir you created) in a finally/teardown block after venv_check_call finishes;
reference model_src, model_dst, venv_check_call and output_path when making
these changes.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 8e95a48d-d9e2-4388-a0e8-d8bb32cb22de

📥 Commits

Reviewing files that changed from the base of the PR and between e73d068 and e6ba49b.

📒 Files selected for processing (8)
  • docs/source/helper.py
  • docs/source/index.rst
  • examples/visual_gen/api_walkthrough.py
  • tests/integration/defs/examples/test_visual_gen.py
  • tests/integration/test_lists/test-db/l0_dgx_b200.yml
  • tests/integration/test_lists/test-db/l0_gh200.yml
  • tests/integration/test_lists/test-db/l0_h100.yml
  • tests/integration/test_lists/test-db/l0_l40s.yml

Comment thread examples/visual_gen/api_walkthrough.py
Comment thread examples/visual_gen/api_walkthrough.py
@tensorrt-cicd
Copy link
Copy Markdown
Collaborator

PR_Github #50757 [ run ] triggered by Bot. Commit: e6ba49b Link to invocation

@tensorrt-cicd
Copy link
Copy Markdown
Collaborator

PR_Github #50757 [ run ] completed with state SUCCESS. Commit: e6ba49b
/LLM/main/L0_MergeRequest_PR pipeline #40236 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

@zhenhuaw-me
Copy link
Copy Markdown
Member Author

/bot run

@tensorrt-cicd
Copy link
Copy Markdown
Collaborator

PR_Github #50788 [ run ] triggered by Bot. Commit: e6ba49b Link to invocation

@tensorrt-cicd
Copy link
Copy Markdown
Collaborator

PR_Github #50788 [ run ] completed with state FAILURE. Commit: e6ba49b
/LLM/main/L0_MergeRequest_PR pipeline #40263 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

@zhenhuaw-me
Copy link
Copy Markdown
Member Author

/bot run

@tensorrt-cicd
Copy link
Copy Markdown
Collaborator

PR_Github #50924 [ run ] triggered by Bot. Commit: e6ba49b Link to invocation

@tensorrt-cicd
Copy link
Copy Markdown
Collaborator

PR_Github #50924 [ run ] completed with state SUCCESS. Commit: e6ba49b
/LLM/main/L0_MergeRequest_PR pipeline #40385 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

  • Please check the failed tests and fix your PR
  • If you cannot view the failures, ask the CI triggerer to share details
  • Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

@zhenhuaw-me
Copy link
Copy Markdown
Member Author

/bot run

@tensorrt-cicd
Copy link
Copy Markdown
Collaborator

PR_Github #51027 [ run ] triggered by Bot. Commit: e6ba49b Link to invocation

@tensorrt-cicd
Copy link
Copy Markdown
Collaborator

PR_Github #51027 [ run ] completed with state SUCCESS. Commit: e6ba49b
/LLM/main/L0_MergeRequest_PR pipeline #40474 completed with status: 'SUCCESS'

CI Report

Link to invocation

Copy link
Copy Markdown
Collaborator

@karljang karljang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Comment thread examples/visual_gen/api_walkthrough.py
@zhenhuaw-me zhenhuaw-me merged commit c3f6d98 into NVIDIA:main Jun 2, 2026
21 checks passed
@zhenhuaw-me zhenhuaw-me deleted the api-example branch June 2, 2026 03:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api-compatible Accepted LLM API contract change that is backwards-compatible Doc <NV>TRTLLM's textual/illustrative materials: API refs, guides, tutorials. Improvement & clarity. VisualGen

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants