Update ds v3 parameters in stress test. #3676

dominicshanshan · 2025-04-18T06:14:57Z

No description provided.

dominicshanshan · 2025-04-18T08:35:56Z

/bot run

tensorrt-cicd · 2025-04-18T08:42:05Z

PR_Github #2756 [ run ] triggered by Bot

tensorrt-cicd · 2025-04-18T10:06:03Z

PR_Github #2756 [ run ] completed with state SUCCESS
/LLM/release-0.19/L0_MergeRequest_PR pipeline #38 completed with status: 'FAILURE'

dominicshanshan · 2025-04-19T03:24:45Z

/bot run

tensorrt-cicd · 2025-04-19T03:30:29Z

PR_Github #2812 [ run ] triggered by Bot

tensorrt-cicd · 2025-04-19T07:13:51Z

PR_Github #2812 [ run ] completed with state SUCCESS
/LLM/release-0.19/L0_MergeRequest_PR pipeline #44 completed with status: 'SUCCESS'

Copilot

Pull Request Overview

This PR updates the ds v3 parameters used in the stress test suite. Key changes include:

Updating test naming conventions in YAML files to remove the "stress-stage-alone" suffix.
Increasing the graceful termination timeout from 10 seconds to 300 seconds.
Introducing ds v3–specific parameters in both performance and server configurations within the stress test definition.

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
tests/integration/test_lists/test-db/l0_a10.yml	Updated test names to align with ds v3 stress test naming conventions.
tests/integration/defs/stress_test/stress_test.py	Modified timeouts and added ds v3–specific performance and server configuration options.

Comments suppressed due to low confidence (1)

tests/integration/defs/stress_test/stress_test.py:399

The new ServerConfig creation block references test_server_config (e.g., test_server_config.port) even though it has not been defined in this block. Consider using the passed server_config or initializing test_server_config before referencing its properties.

if "DeepSeek-V3" in config.model_dir:

Signed-off-by: Wangshanshan <30051912+dominicshanshan@users.noreply.github.com>

dominicshanshan · 2025-04-19T07:19:13Z

/bot reuse-pipeline

tensorrt-cicd · 2025-04-19T07:25:12Z

PR_Github #2827 [ reuse-pipeline ] triggered by Bot

tensorrt-cicd · 2025-04-19T07:30:36Z

PR_Github #2827 [ reuse-pipeline ] completed with state SUCCESS
Reusing PR_Github #2812 for commit 172823c

dominicshanshan · 2025-04-21T04:56:44Z

@joyang-nv Jonas, could you help me to auto-merge into release/0.19?

Signed-off-by: ZhanruiSunCh <184402041+ZhanruiSunCh@users.noreply.github.com> test: add test cases for 0.19 release (NVIDIA#3608) * fix test name Signed-off-by: Ivy Zhang <25222398+crazydemo@users.noreply.github.com> * add quickstart test for nemotron-ultra Signed-off-by: Ivy Zhang <25222398+crazydemo@users.noreply.github.com> * add rcca multi-node test case for deepseek-v3 Signed-off-by: Ivy Zhang <yanzh@nvidia.com> * add rcca info Signed-off-by: Ivy Zhang <25222398+crazydemo@users.noreply.github.com> --------- Signed-off-by: Ivy Zhang <25222398+crazydemo@users.noreply.github.com> Signed-off-by: Ivy Zhang <yanzh@nvidia.com> squash (NVIDIA#3642) Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> fix: nvbugs/5187237: fix deterministic mode crash (NVIDIA#3448) * nvbugs/5187237 nvbugs/5112075: fix deterministic mode error * remove waive Signed-off-by: Xiwen Yu <13230610+VALLIS-NERIA@users.noreply.github.com> * Revert "remove waive" This reverts commit 0bf5486d19906d692bfb7a6262333c296b0087ac. Signed-off-by: Xiwen Yu <13230610+VALLIS-NERIA@users.noreply.github.com> * revert ar fusion Signed-off-by: Xiwen Yu <13230610+VALLIS-NERIA@users.noreply.github.com> --------- Signed-off-by: Xiwen Yu <13230610+VALLIS-NERIA@users.noreply.github.com> update fp8 doc (NVIDIA#3647) Signed-off-by: taoli <litaotju@users.noreply.github.com> Co-authored-by: taoli <litaotju@users.noreply.github.com> tests: change qa perf test to trtllm-bench (NVIDIA#3619) Signed-off-by: Ruodi <200874449+ruodil@users.noreply.github.com> Co-authored-by: Larry <197874197+LarryXFly@users.noreply.github.com> fix: FP8 quantized lm_head (NvBug 5214229) (NVIDIA#3567) Signed-off-by: Enwei Zhu <21126786+syuoni@users.noreply.github.com> infra: Add PR approval protection for the release branch (NVIDIA#3634) Signed-off-by: Yanchao Lu <yanchaol@nvidia.com> fix: nvbugs/5231298: pytorch allreduce issue (NVIDIA#3673) Signed-off-by: Xiwen Yu <13230610+VALLIS-NERIA@users.noreply.github.com> Fix: nvbugs/5222698 variable not defined (NVIDIA#3630) * Fix: nvbugs/5222698 variable not defined Signed-off-by: Zongfei Jing <20381269+zongfeijing@users.noreply.github.com> * Tidy code Signed-off-by: Zongfei Jing <20381269+zongfeijing@users.noreply.github.com> --------- Signed-off-by: Zongfei Jing <20381269+zongfeijing@users.noreply.github.com> test:sync waives.txt from main branch by disabling test_perf/gpt_350m-cppmanager case (NVIDIA#3685) Signed-off-by: nv-guomingz <37257613+nv-guomingz@users.noreply.github.com> test:restore fp8 kv cache testing for L0 (NVIDIA#3671) Signed-off-by: nv-guomingz <37257613+nv-guomingz@users.noreply.github.com> doc: Update DeepSeek perf docs (NVIDIA#3693) * Update DeepSeek perf docs Signed-off-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com> * update Signed-off-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com> --------- Signed-off-by: Kaiyu Xie <26294424+kaiyux@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> tests: waive test_llm_multi_node (NVIDIA#3664) Signed-off-by: junq <22017000+QiJune@users.noreply.github.com> fix: update test_user_buffers_mm_add_prologue atol (NVIDIA#3711) Signed-off-by: Jin Li <59594262+liji-nv@users.noreply.github.com> Fix: cherry-pick hmac encryption from main branch (NVIDIA#3635) * security fix cherry-pick changes from main Signed-off-by: Yibin Li <109242046+yibinl-nvidia@users.noreply.github.com> * fix hmac in remote mpi session (NVIDIA#3649) Signed-off-by: Yan Chunwei <328693+Superjomn@users.noreply.github.com> --------- Signed-off-by: Yibin Li <109242046+yibinl-nvidia@users.noreply.github.com> Signed-off-by: Yan Chunwei <328693+Superjomn@users.noreply.github.com> Co-authored-by: Yan Chunwei <328693+Superjomn@users.noreply.github.com> Un-waive DS-V3-Lite tests. (NVIDIA#3621) Signed-off-by: Tracin <10434017+Tracin@users.noreply.github.com> fix: FP8 kv accuracy (NVIDIA#3675) * fix FP8 kv accuracy Signed-off-by: Dylan Chen <191843203+DylanChen-NV@users.noreply.github.com> * update doc Signed-off-by: Dylan Chen <191843203+DylanChen-NV@users.noreply.github.com> --------- Signed-off-by: Dylan Chen <191843203+DylanChen-NV@users.noreply.github.com> Fix script options for engines. (NVIDIA#3622) Signed-off-by: Tracin <10434017+Tracin@users.noreply.github.com> unwaive multi-node test (NVIDIA#3721) Signed-off-by: Superjomn <328693+Superjomn@users.noreply.github.com> chore : Split more tests out of gpt tests (NVIDIA#3524) (NVIDIA#3674) Signed-off-by: peaceh <103117813+peaceh-nv@users.noreply.github.com> doc:add torch examples link into torch backend documentation (NVIDIA#3749) Signed-off-by: nv-guomingz <37257613+nv-guomingz@users.noreply.github.com> Co-authored-by: nv-guomingz <37257613+nv-guomingz@users.noreply.github.com> test: Get Eagle tests working (NVIDIA#3593) (NVIDIA#3722) Signed-off-by: Balaram Buddharaju <169953907+brb-nv@users.noreply.github.com> Co-authored-by: brb-nv <169953907+brb-nv@users.noreply.github.com> Waive L0 test (NVIDIA#3756) Signed-off-by: Yiqing Yan <yiqingy@nvidia.com> waive failed case in perf test, change default max_batch_size to 512 and write config.json to output log (NVIDIA#3656) Signed-off-by: Ruodi <200874449+ruodil@users.noreply.github.com> Signed-off-by: Larry <197874197+LarryXFly@users.noreply.github.com> Co-authored-by: Larry <197874197+LarryXFly@users.noreply.github.com> Update ds v3 parameters in stress test. (NVIDIA#3676) waive gemma on L20 (NVIDIA#3766) Signed-off-by: Ivy Zhang <25222398+crazydemo@users.noreply.github.com> https://nvbugs/5141291: Fix convert.py script for Qwen model. (NVIDIA#3758) Include Qwen2VLDecoderLayer in the smooth_qwen2_model function. Signed-off-by: Yukun He <23156053+hyukn@users.noreply.github.com> fix: PP4 fixes and cleanup (NVIDIA#3688) Signed-off-by: Anurag Mukkara <134339030+amukkara@users.noreply.github.com> Co-authored-by: Sharan Chetlur <116769508+schetlur-nv@users.noreply.github.com> remove benchmark test list (NVIDIA#3643) Signed-off-by: Ivy Zhang <25222398+crazydemo@users.noreply.github.com> skip disagg deepseek test if sm!=90 (NVIDIA#3720) Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com> test: skip failed cases on B200 (NVIDIA#3710) * add skip condition to tests Signed-off-by: xinhe-nv <200704525+xinhe-nv@users.noreply.github.com> * fix error Signed-off-by: xinhe-nv <200704525+xinhe-nv@users.noreply.github.com> --------- Signed-off-by: xinhe-nv <200704525+xinhe-nv@users.noreply.github.com> test: [nvbug: 5234494] skip_pre_ada for fp8 cases (NVIDIA#3718) * skip_pre_ada for fp8 cases Signed-off-by: Ivy Zhang <25222398+crazydemo@users.noreply.github.com> * update Signed-off-by: Ivy Zhang <25222398+crazydemo@users.noreply.github.com> * update after rebase Signed-off-by: Ivy Zhang <25222398+crazydemo@users.noreply.github.com> --------- Signed-off-by: Ivy Zhang <25222398+crazydemo@users.noreply.github.com> add know issue to deepseek doc. (NVIDIA#3800) Signed-off-by: Fanrong Li <23290157+lfr-0531@users.noreply.github.com> Fix ModelOpt Mixtral AWQ OOM (NVIDIA#3714) (NVIDIA#3761) Signed-off-by: Barry Kang <43644113+Barry-Delaney@users.noreply.github.com> Co-authored-by: Larry <197874197+LarryXFly@users.noreply.github.com> Waive L0 tests (NVIDIA#3826) Signed-off-by: Yiqing Yan <yiqingy@nvidia.com> fix: Reduce memory usage in fused moe op associated with AutoTuning and fix moe fallback issue. (NVIDIA#3793) * Reduce memory usage in fused moe op associated with AutoTuning. * Replace pre-defined bucket size strategy with a generating function based on the tune_max_num_tokens. * Add free_memory logic of workspace in min_latency_mode fused moe path. Signed-off-by: Yukun He <23156053+hyukn@users.noreply.github.com> * Fix fused_moe fallback issue. (NVIDIA#3652) min_latency_mode is only set to False during warmup phase. Thus when it becomes true during inference, all tactics fall back to the default one and thus cause perf regression. Signed-off-by: Yukun He <23156053+hyukn@users.noreply.github.com> --------- Signed-off-by: Yukun He <23156053+hyukn@users.noreply.github.com> [doc] Better document for Draft-Target-Model (DTM) speculative decoding (NVIDIA#3797) Signed-off-by: wili-65535 <wili-65535@users.noreply.github.com> Signed-off-by: Dom Brown <3886319+DomBrown@users.noreply.github.com> Fix pre-commit Signed-off-by: Dom Brown <3886319+DomBrown@users.noreply.github.com> Fix again Signed-off-by: Dom Brown <3886319+DomBrown@users.noreply.github.com> Address some review comments for the MI Signed-off-by: Dom Brown <3886319+DomBrown@users.noreply.github.com>

test: add test cases for 0.19 release (#3608) * fix test name * add quickstart test for nemotron-ultra * add rcca multi-node test case for deepseek-v3 * add rcca info --------- squash (#3642) fix: nvbugs/5187237: fix deterministic mode crash (#3448) * nvbugs/5187237 nvbugs/5112075: fix deterministic mode error * remove waive * Revert "remove waive" This reverts commit 0bf5486d19906d692bfb7a6262333c296b0087ac. * revert ar fusion --------- update fp8 doc (#3647) tests: change qa perf test to trtllm-bench (#3619) fix: FP8 quantized lm_head (NvBug 5214229) (#3567) infra: Add PR approval protection for the release branch (#3634) fix: nvbugs/5231298: pytorch allreduce issue (#3673) Fix: nvbugs/5222698 variable not defined (#3630) * Fix: nvbugs/5222698 variable not defined * Tidy code --------- test:sync waives.txt from main branch by disabling test_perf/gpt_350m-cppmanager case (#3685) test:restore fp8 kv cache testing for L0 (#3671) doc: Update DeepSeek perf docs (#3693) * Update DeepSeek perf docs * update * Apply suggestions from code review --------- tests: waive test_llm_multi_node (#3664) fix: update test_user_buffers_mm_add_prologue atol (#3711) Fix: cherry-pick hmac encryption from main branch (#3635) * security fix cherry-pick changes from main * fix hmac in remote mpi session (#3649) --------- Un-waive DS-V3-Lite tests. (#3621) fix: FP8 kv accuracy (#3675) * fix FP8 kv accuracy * update doc --------- Fix script options for engines. (#3622) unwaive multi-node test (#3721) chore : Split more tests out of gpt tests (#3524) (#3674) doc:add torch examples link into torch backend documentation (#3749) test: Get Eagle tests working (#3593) (#3722) Waive L0 test (#3756) waive failed case in perf test, change default max_batch_size to 512 and write config.json to output log (#3656) Update ds v3 parameters in stress test. (#3676) waive gemma on L20 (#3766) https://nvbugs/5141291: Fix convert.py script for Qwen model. (#3758) Include Qwen2VLDecoderLayer in the smooth_qwen2_model function. fix: PP4 fixes and cleanup (#3688) remove benchmark test list (#3643) skip disagg deepseek test if sm!=90 (#3720) test: skip failed cases on B200 (#3710) * add skip condition to tests * fix error --------- test: [nvbug: 5234494] skip_pre_ada for fp8 cases (#3718) * skip_pre_ada for fp8 cases * update * update after rebase --------- add know issue to deepseek doc. (#3800) Fix ModelOpt Mixtral AWQ OOM (#3714) (#3761) Waive L0 tests (#3826) fix: Reduce memory usage in fused moe op associated with AutoTuning and fix moe fallback issue. (#3793) * Reduce memory usage in fused moe op associated with AutoTuning. * Replace pre-defined bucket size strategy with a generating function based on the tune_max_num_tokens. * Add free_memory logic of workspace in min_latency_mode fused moe path. * Fix fused_moe fallback issue. (#3652) min_latency_mode is only set to False during warmup phase. Thus when it becomes true during inference, all tactics fall back to the default one and thus cause perf regression. --------- [doc] Better document for Draft-Target-Model (DTM) speculative decoding (#3797) Fix pre-commit Fix again Address some review comments for the MI Signed-off-by: Dom Brown <3886319+DomBrown@users.noreply.github.com> Co-authored-by: Zhanrui Sun <184402041+ZhanruiSunCh@users.noreply.github.com>

dominicshanshan requested a review from a team as a code owner April 18, 2025 06:14

dominicshanshan force-pushed the user/shanshan/update_ds_v3_parameters_stress_test branch from 07b9313 to 2dee99d Compare April 18, 2025 07:00

dominicshanshan requested review from joyang-nv, litaotju and tongyuantongyu April 18, 2025 07:14

dominicshanshan force-pushed the user/shanshan/update_ds_v3_parameters_stress_test branch from 2dee99d to 30c63cb Compare April 19, 2025 03:21

dominicshanshan requested a review from Copilot April 19, 2025 07:17

Copilot AI reviewed Apr 19, 2025

View reviewed changes

Update ds v3 parameters in stress test.

172823c

Signed-off-by: Wangshanshan <30051912+dominicshanshan@users.noreply.github.com>

dominicshanshan force-pushed the user/shanshan/update_ds_v3_parameters_stress_test branch from 30c63cb to 172823c Compare April 19, 2025 07:18

joyang-nv approved these changes Apr 19, 2025

View reviewed changes

litaotju approved these changes Apr 21, 2025

View reviewed changes

litaotju merged commit 792b71f into NVIDIA:release/0.19 Apr 22, 2025
3 checks passed

dominicshanshan deleted the user/shanshan/update_ds_v3_parameters_stress_test branch April 22, 2025 10:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Update ds v3 parameters in stress test. #3676

Update ds v3 parameters in stress test. #3676

Uh oh!

dominicshanshan commented Apr 18, 2025

Uh oh!

dominicshanshan commented Apr 18, 2025

Uh oh!

tensorrt-cicd commented Apr 18, 2025

Uh oh!

tensorrt-cicd commented Apr 18, 2025

Uh oh!

dominicshanshan commented Apr 19, 2025

Uh oh!

tensorrt-cicd commented Apr 19, 2025

Uh oh!

tensorrt-cicd commented Apr 19, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

dominicshanshan commented Apr 19, 2025

Uh oh!

tensorrt-cicd commented Apr 19, 2025

Uh oh!

tensorrt-cicd commented Apr 19, 2025

Uh oh!

dominicshanshan commented Apr 21, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Update ds v3 parameters in stress test. #3676

Update ds v3 parameters in stress test. #3676

Uh oh!

Conversation

dominicshanshan commented Apr 18, 2025

Uh oh!

dominicshanshan commented Apr 18, 2025

Uh oh!

tensorrt-cicd commented Apr 18, 2025

Uh oh!

tensorrt-cicd commented Apr 18, 2025

Uh oh!

dominicshanshan commented Apr 19, 2025

Uh oh!

tensorrt-cicd commented Apr 19, 2025

Uh oh!

tensorrt-cicd commented Apr 19, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

dominicshanshan commented Apr 19, 2025

Uh oh!

tensorrt-cicd commented Apr 19, 2025

Uh oh!

tensorrt-cicd commented Apr 19, 2025

Uh oh!

dominicshanshan commented Apr 21, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants