[None][test] Enable session prefetch for all test stages#16770
Open
sunnyqgg wants to merge 1 commit into
Open
Conversation
Signed-off-by: qgai <qgai@nvidia.com>
sunnyqgg
requested review from
JunyiXu-nv,
brnguyen2,
schetlur-nv and
tongyuantongyu
July 23, 2026 02:59
Contributor
WalkthroughSession prefetching now enables by default outside xdist workers, with ChangesSession prefetch default activation
Estimated code review effort: 2 (Simple) | ~10 minutes Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
tests/unittest/llmapi/test_session_prefetcher.py (1)
96-106: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAnnotate the new test functions.
Add parameter and return annotations to comply with the repository’s Python requirements, for example
stage_name: str | None,monkeypatch: pytest.MonkeyPatch, and-> None.As per coding guidelines, every Python function must be annotated.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/unittest/llmapi/test_session_prefetcher.py` around lines 96 - 106, Annotate the new test functions test_enabled_by_default_in_all_stages and test_explicit_env_overrides_default with the required parameter and return types, including pytest.MonkeyPatch, stage_name: str | None where applicable, and -> None.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tests/unittest/llmapi/test_session_prefetcher.py`:
- Around line 86-111: Add a regression test for SessionPrefetcher that sets
PYTEST_XDIST_WORKER while leaving prefetch otherwise enabled, and assert
SessionPrefetcher().enabled is false. Keep the existing non-xdist default and
explicit TRTLLM_TEST_PREFETCH_SESSION override tests unchanged.
---
Nitpick comments:
In `@tests/unittest/llmapi/test_session_prefetcher.py`:
- Around line 96-106: Annotate the new test functions
test_enabled_by_default_in_all_stages and test_explicit_env_overrides_default
with the required parameter and return types, including pytest.MonkeyPatch,
stage_name: str | None where applicable, and -> None.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 6a5680b7-98df-42d6-b19e-d1e7091817e8
📒 Files selected for processing (2)
tests/test_common/session_prefetcher.pytests/unittest/llmapi/test_session_prefetcher.py
Comment on lines
+86
to
+111
| @pytest.mark.parametrize( | ||
| "stage_name", | ||
| [ | ||
| None, | ||
| "A10-PyTorch-2", | ||
| "DGX_H100-4_GPUs-PyTorch-DeepSeek-1", | ||
| "DGX_B200-PyTorch-4", | ||
| "Any-Future-Stage", | ||
| ], | ||
| ) | ||
| def test_enabled_by_default_in_all_stages(monkeypatch, stage_name): | ||
| monkeypatch.delenv("TRTLLM_TEST_PREFETCH_SESSION", raising=False) | ||
| monkeypatch.delenv("PYTEST_XDIST_WORKER", raising=False) | ||
| monkeypatch.setenv("stageName", "A10-PyTorch-2") | ||
| assert SessionPrefetcher().enabled | ||
| monkeypatch.setenv("stageName", "DGX_H100-4_GPUs-PyTorch-DeepSeek-1") | ||
| if stage_name is None: | ||
| monkeypatch.delenv("stageName", raising=False) | ||
| else: | ||
| monkeypatch.setenv("stageName", stage_name) | ||
| assert SessionPrefetcher().enabled | ||
|
|
||
|
|
||
| def test_canary_disabled_elsewhere(monkeypatch): | ||
| monkeypatch.delenv("TRTLLM_TEST_PREFETCH_SESSION", raising=False) | ||
| def test_explicit_env_overrides_default(monkeypatch): | ||
| monkeypatch.delenv("PYTEST_XDIST_WORKER", raising=False) | ||
| monkeypatch.setenv("stageName", "DGX_B200-PyTorch-4") | ||
| assert not SessionPrefetcher().enabled | ||
| monkeypatch.delenv("stageName", raising=False) # local run, no stage | ||
| monkeypatch.setenv("TRTLLM_TEST_PREFETCH_SESSION", "0") | ||
| assert not SessionPrefetcher().enabled | ||
|
|
||
|
|
||
| def test_explicit_env_overrides_canary_gate(monkeypatch): | ||
| monkeypatch.delenv("PYTEST_XDIST_WORKER", raising=False) | ||
| monkeypatch.setenv("stageName", "DGX_B200-PyTorch-4") # not a canary stage | ||
| monkeypatch.setenv("TRTLLM_TEST_PREFETCH_SESSION", "1") | ||
| assert SessionPrefetcher().enabled # manual opt-in anywhere | ||
| monkeypatch.setenv("stageName", "A10-PyTorch-1") # canary stage | ||
| monkeypatch.setenv("TRTLLM_TEST_PREFETCH_SESSION", "0") | ||
| assert not SessionPrefetcher().enabled # kill switch beats the canary | ||
| assert SessionPrefetcher().enabled |
Contributor
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Add an xdist regression test.
Both tests explicitly remove PYTEST_XDIST_WORKER, so they do not verify the preserved contract that xdist workers remain disabled even when prefetch is otherwise enabled.
Proposed test
+def test_disabled_in_xdist_even_when_enabled(monkeypatch: pytest.MonkeyPatch) -> None:
+ monkeypatch.setenv("PYTEST_XDIST_WORKER", "gw0")
+ monkeypatch.setenv("TRTLLM_TEST_PREFETCH_SESSION", "1")
+ assert not SessionPrefetcher().enabledThe PR objective explicitly preserves xdist disablement.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| @pytest.mark.parametrize( | |
| "stage_name", | |
| [ | |
| None, | |
| "A10-PyTorch-2", | |
| "DGX_H100-4_GPUs-PyTorch-DeepSeek-1", | |
| "DGX_B200-PyTorch-4", | |
| "Any-Future-Stage", | |
| ], | |
| ) | |
| def test_enabled_by_default_in_all_stages(monkeypatch, stage_name): | |
| monkeypatch.delenv("TRTLLM_TEST_PREFETCH_SESSION", raising=False) | |
| monkeypatch.delenv("PYTEST_XDIST_WORKER", raising=False) | |
| monkeypatch.setenv("stageName", "A10-PyTorch-2") | |
| assert SessionPrefetcher().enabled | |
| monkeypatch.setenv("stageName", "DGX_H100-4_GPUs-PyTorch-DeepSeek-1") | |
| if stage_name is None: | |
| monkeypatch.delenv("stageName", raising=False) | |
| else: | |
| monkeypatch.setenv("stageName", stage_name) | |
| assert SessionPrefetcher().enabled | |
| def test_canary_disabled_elsewhere(monkeypatch): | |
| monkeypatch.delenv("TRTLLM_TEST_PREFETCH_SESSION", raising=False) | |
| def test_explicit_env_overrides_default(monkeypatch): | |
| monkeypatch.delenv("PYTEST_XDIST_WORKER", raising=False) | |
| monkeypatch.setenv("stageName", "DGX_B200-PyTorch-4") | |
| assert not SessionPrefetcher().enabled | |
| monkeypatch.delenv("stageName", raising=False) # local run, no stage | |
| monkeypatch.setenv("TRTLLM_TEST_PREFETCH_SESSION", "0") | |
| assert not SessionPrefetcher().enabled | |
| def test_explicit_env_overrides_canary_gate(monkeypatch): | |
| monkeypatch.delenv("PYTEST_XDIST_WORKER", raising=False) | |
| monkeypatch.setenv("stageName", "DGX_B200-PyTorch-4") # not a canary stage | |
| monkeypatch.setenv("TRTLLM_TEST_PREFETCH_SESSION", "1") | |
| assert SessionPrefetcher().enabled # manual opt-in anywhere | |
| monkeypatch.setenv("stageName", "A10-PyTorch-1") # canary stage | |
| monkeypatch.setenv("TRTLLM_TEST_PREFETCH_SESSION", "0") | |
| assert not SessionPrefetcher().enabled # kill switch beats the canary | |
| assert SessionPrefetcher().enabled | |
| `@pytest.mark.parametrize`( | |
| "stage_name", | |
| [ | |
| None, | |
| "A10-PyTorch-2", | |
| "DGX_H100-4_GPUs-PyTorch-DeepSeek-1", | |
| "DGX_B200-PyTorch-4", | |
| "Any-Future-Stage", | |
| ], | |
| ) | |
| def test_enabled_by_default_in_all_stages(monkeypatch, stage_name): | |
| monkeypatch.delenv("TRTLLM_TEST_PREFETCH_SESSION", raising=False) | |
| monkeypatch.delenv("PYTEST_XDIST_WORKER", raising=False) | |
| if stage_name is None: | |
| monkeypatch.delenv("stageName", raising=False) | |
| else: | |
| monkeypatch.setenv("stageName", stage_name) | |
| assert SessionPrefetcher().enabled | |
| def test_explicit_env_overrides_default(monkeypatch): | |
| monkeypatch.delenv("PYTEST_XDIST_WORKER", raising=False) | |
| monkeypatch.setenv("TRTLLM_TEST_PREFETCH_SESSION", "0") | |
| assert not SessionPrefetcher().enabled | |
| monkeypatch.setenv("TRTLLM_TEST_PREFETCH_SESSION", "1") | |
| assert SessionPrefetcher().enabled | |
| def test_disabled_in_xdist_even_when_enabled(monkeypatch: pytest.MonkeyPatch) -> None: | |
| monkeypatch.setenv("PYTEST_XDIST_WORKER", "gw0") | |
| monkeypatch.setenv("TRTLLM_TEST_PREFETCH_SESSION", "1") | |
| assert not SessionPrefetcher().enabled |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@tests/unittest/llmapi/test_session_prefetcher.py` around lines 86 - 111, Add
a regression test for SessionPrefetcher that sets PYTEST_XDIST_WORKER while
leaving prefetch otherwise enabled, and assert SessionPrefetcher().enabled is
false. Keep the existing non-xdist default and explicit
TRTLLM_TEST_PREFETCH_SESSION override tests unchanged.
Collaborator
Author
|
/bot run --disable-fail-fast |
Collaborator
|
PR_Github #61174 [ run ] triggered by Bot. Commit: |
Collaborator
|
PR_Github #61174 [ run ] completed with state
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
TRTLLM_TEST_PREFETCH_SESSION=0as the kill switch and keep prefetch disabled in pytest-xdist workers.Changes
tests/test_common/session_prefetcher.py: remove the canary stage allowlist and enable prefetch by default for every test stage.tests/unittest/llmapi/test_session_prefetcher.py: cover canary, non-canary, future, and local/no-stage execution plus explicit env overrides.Test plan
python3 -m pytest llmapi/test_session_prefetcher.py -v -p no:cacheproviderin the TensorRT-LLM development container (42 passed)git diff --checkDev Engineer Review
TRTLLM_TEST_PREFETCH_SESSION=0as an explicit kill switch and continued disabling prefetch in pytest-xdist workers.git diff --check.QA Engineer Review
Test functions added:
test_enabled_by_default_in_all_stagestest_explicit_env_overrides_defaultTest functions removed:
test_canary_enabled_on_canary_stage_groupstest_canary_disabled_elsewheretest_explicit_env_overrides_canary_gateThese test-code changes are not listed in
tests/integration/test_lists/test-db or QA files.Verdict: needs follow-up — CI/manual QA test-list coverage is not represented for the updated test functions.