[None][refactor] Clean up model paths and remove deprecated configurations in performance tests - #17057
Conversation
…n model YAML (NVIDIA#16935) Signed-off-by: yufeiwu-nv <230315618+yufeiwu-nv@users.noreply.github.com>
…tions in performance tests - Removed unused model paths from _model_paths.py. - Updated sampler_options_config.py to reflect changes in model labels. - Cleaned up pytorch_model_config.py by removing obsolete DeepSeek R1 model configurations. These changes streamline the model configuration files and ensure they only contain relevant entries. Signed-off-by: yufeiwu-nv <230315618+yufeiwu-nv@users.noreply.github.com>
WalkthroughPerformance integration definitions update model path mappings, backend overrides, benchmark pattern configurations, and sampler-label matching for selected model variants. ChangesPerformance configuration
Estimated code review effort: 4 (Complex) | ~40 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1🛠️ Fix failing CI checks 💡
🧪 Generate unit tests (beta)
Comment |
Signed-off-by: yufeiwu-nv <230315618+yufeiwu-nv@users.noreply.github.com> # Conflicts: # tests/integration/defs/perf/_model_paths.py
|
/bot skip --comment "only test modify" |
|
PR_Github #62738 [ skip ] triggered by Bot. Commit: |
|
PR_Github #62738 [ skip ] completed with state |
|
/bot skip --comment "only test modify" |
|
PR_Github #62752 [ skip ] triggered by Bot. Commit: |
|
PR_Github #62752 [ skip ] completed with state |
|
The removed-key audit is clean. I checked all 14 keys dropped from What gives me pause is that this isn't only deletion. Three of the replacement blocks swap an explicit label list for a bare model-name pattern, and matching is substring, not equality —
The sampler change has the same shape: the retained label now canonicalizes to what In every one of these the test identity is unchanged, so they keep their existing recorded perf baselines while their runtime configuration changes underneath. That's what I'd want covered before this lands — either refreshed baselines or a before/after run showing the current thresholds still hold. If the deltas are known to be within noise, say so and I'm happy; I just can't tell that from the diff. To be clear, the |
|
/bot skip --comment "only test modify" |
|
PR_Github #62930 [ skip ] triggered by Bot. Commit: |
|
PR_Github #62930 [ skip ] completed with state |
Dev Engineer Review
tests/integration/defs/perf/_model_paths.py(including various Llama/Qwen/Phi-Bielik/Nemotron/DeepSeek R1 variants).tests/integration/defs/perf/pytorch_model_config.py:moe_config.backend: WIDEEPoverride while keeping attention DP enablement.deepseek_r1_fp8and addeddeepseek_r1_0528_fp8to ensure the DEEPGEMM MoE backend override applies on SM100+.tests/integration/defs/perf/sampler_options_config.py:model_labelmatching is now based on exact equality toPerfTestConfig.to_string()output.QA Engineer Review
tests/:tests/integration/defs/perf/_model_paths.pytests/integration/defs/perf/pytorch_model_config.pytests/integration/defs/perf/sampler_options_config.pytests/integration/test_lists/.Description
Test Coverage
PR Checklist
Please review the following before submitting your PR:
PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
If PR introduces API changes, an appropriate PR label is added - either
api-compatibleorapi-breaking. Forapi-breaking, includeBREAKINGin the PR title.Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.
GitHub Bot Help
To see a list of available CI bot commands, please comment
/bot help.