[TRTLLM-11358][test] Add trtllm-serve e2e tests for FLUX#12153
Conversation
|
/bot run |
📝 WalkthroughWalkthroughThe PR adds FLUX.1 and FLUX.2 text-to-image model support to an E2E test suite, introducing model path constants, two new test classes with synchronous text-to-image generation tests, and base64 image validation. Existing WAN tests remain unchanged with conditional skip logic. Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes 🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@tests/unittest/_torch/visual_gen/test_trtllm_serve_e2e.py`:
- Around line 503-521: Update the two optional-parameter tests
(test_t2i_sync_with_optional_params and the similar test at lines 571-589) to
validate the returned b64_json payload contents rather than only its presence:
copy the decode/size/assertions used in test_t2i_sync_b64 (decode base64,
convert to bytes, assert non-empty and expected image dimensions or valid image
format) and apply them to data["data"][0]["b64_json"] so malformed payloads fail
the test.
- Around line 17-26: Update the example pytest commands so they point to the
actual test module path used in this repo; replace occurrences of
tests/visual_gen/test_trtllm_serve_e2e.py with
tests/unittest/_torch/visual_gen/test_trtllm_serve_e2e.py in all example lines
(the commands that run TestWanTextToVideo, TestWanImageToVideo,
TestFlux1TextToImage, and TestFlux2TextToImage) so the provided examples execute
the correct module.
- Around line 477-517: Add a module-level timeout constant (e.g.,
REQUEST_TIMEOUT_S = 60) and pass it as timeout=REQUEST_TIMEOUT_S to every
requests.get/requests.post call in the test classes; specifically update the
requests callsites in the test_health, test_t2i_sync_b64, and
test_t2i_sync_with_optional_params methods (and the corresponding methods in the
other FLUX test class) so each HTTP call includes timeout=REQUEST_TIMEOUT_S to
avoid indefinite hangs.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
Run ID: e4860d57-9de2-4de1-b7af-00a797e65f5a
📒 Files selected for processing (1)
tests/unittest/_torch/visual_gen/test_trtllm_serve_e2e.py
|
PR_Github #38708 [ run ] triggered by Bot. Commit: |
|
PR_Github #38708 [ run ] completed with state |
|
/bot run |
|
PR_Github #39215 [ run ] triggered by Bot. Commit: |
|
PR_Github #39215 [ run ] completed with state
|
|
/bot run |
|
PR_Github #39356 [ run ] triggered by Bot. Commit: |
|
PR_Github #39356 [ run ] completed with state
|
|
/bot run |
|
PR_Github #39417 [ run ] triggered by Bot. Commit: |
|
PR_Github #39417 [ run ] completed with state
|
JunyiXu-nv
force-pushed
the
user/junyix/trtllm-11358-flux-serve-e2e-tests
branch
from
ba29d6f to
af90d91
Compare
|
/bot run |
|
PR_Github #39426 [ run ] triggered by Bot. Commit: |
|
PR_Github #39426 [ run ] completed with state
|
JunyiXu-nv
requested review from
QiJune and
hchings
and removed request for
hchings
- Add TestFlux1TextToImage for FLUX.1-dev text-to-image generation - Add TestFlux2TextToImage for FLUX.2-dev text-to-image generation - Tests cover health check, sync image generation via /v1/images/generations, and optional params (guidance_scale, negative_prompt, seed, size) - Follow existing WAN e2e test patterns with RemoteVisualGenServer - Model paths configurable via FLUX1_MODEL_PATH / FLUX2_MODEL_PATH env vars Signed-off-by: Junyi Xu <219237550+JunyiXu-nv@users.noreply.github.com>
- Remove _model_available skipif from all test classes so missing models fail loudly on CI instead of being silently skipped - Simplify FLUX model paths to use _llm_models_root() directly (same pattern as WAN), removing unnecessary env-var overrides - Fix pytest command paths in docstring to match actual module location - Add REQUEST_TIMEOUT_S (600s) to all FLUX HTTP requests to prevent indefinite hangs - Extract _assert_b64_image_response() helper and apply it to all b64_json response validations including optional-param tests Signed-off-by: JunyiXu-nv <219237550+JunyiXu-nv@users.noreply.github.com>
- MediaStorage._to_pil_image() now squeezes 4D tensors (B, H, W, C) to 3D (H, W, C) before calling Image.fromarray(), fixing "Cannot handle this data type: (1, 1, 512, 3), |u1" error when FLUX pipelines return batched image tensors via b64_json response. Signed-off-by: JunyiXu-nv <219237550+JunyiXu-nv@users.noreply.github.com>
JunyiXu-nv
force-pushed
the
user/junyix/trtllm-11358-flux-serve-e2e-tests
branch
from
2c10eb9 to
6ca25db
Compare
|
/bot run |
|
PR_Github #39560 [ run ] triggered by Bot. Commit: |
|
PR_Github #39560 [ run ] completed with state |
4 participants
Summary by CodeRabbit
Description
FLUX is now supported by VisualGen for text-to-image generation. This PR adds
end-to-end tests for the
trtllm-servestack to protect FLUX functionalitythrough the serve API.
Changes:
TestFlux1TextToImagetest class for FLUX.1-dev modelTestFlux2TextToImagetest class for FLUX.2-dev modelPOST /v1/images/generationswith base64 responseFLUX1_MODEL_PATH/FLUX2_MODEL_PATHenv vars,defaulting to
{LLM_MODELS_ROOT}/FLUX.1-devand{LLM_MODELS_ROOT}/FLUX.2-devTest Coverage
tests/unittest/_torch/visual_gen/test_trtllm_serve_e2e.py::TestFlux1TextToImage::test_healthtests/unittest/_torch/visual_gen/test_trtllm_serve_e2e.py::TestFlux1TextToImage::test_t2i_sync_b64tests/unittest/_torch/visual_gen/test_trtllm_serve_e2e.py::TestFlux1TextToImage::test_t2i_sync_with_optional_paramstests/unittest/_torch/visual_gen/test_trtllm_serve_e2e.py::TestFlux2TextToImage::test_healthtests/unittest/_torch/visual_gen/test_trtllm_serve_e2e.py::TestFlux2TextToImage::test_t2i_sync_b64tests/unittest/_torch/visual_gen/test_trtllm_serve_e2e.py::TestFlux2TextToImage::test_t2i_sync_with_optional_paramsTest collection verified: all 16 tests (10 existing + 6 new) collect successfully.
PR Checklist
Please review the following before submitting your PR:
PR description clearly explains what and why.
PR Follows TRT-LLM CODING GUIDELINES.
Test cases are provided for new code paths.
Please check this after reviewing the above items as appropriate for this PR.