[CI] 【Hackathon 9th Sprint No.18】NO.18 功能模块单测补充 #5064

xunyoyo · 2025-11-15T11:20:33Z

This file contains unit tests for the DeepEP buffer helpers and runners, including various test cases for buffer allocation, cleanup, and dispatching processes.

Motivation

NO.18 功能模块 fastdeploy/model_executor/layers/moe/ep.py 单测补充

Modifications

add tests/model_executor/test_ep.py

Usage or Command

tests/model_executor/test_ep.py:

python -m coverage run -m unittest tests.model_executor.test_ep \
&& python -m coverage report -m --include='fastdeploy/model_executor/layers/moe/ep.py'

Accuracy Tests

旧覆盖

File	Stmts	Miss	Branch	BrPart	Cover(%)	Missing
fastdeploy/model_executor/layers/moe/ep.py	208	208	58	0	0	17-646

tests/model_executor/test_ep.py:

Name                                         Stmts   Miss  Cover   Missing
--------------------------------------------------------------------------
fastdeploy/model_executor/layers/moe/ep.py     208     32    85%   25-26, 44-45, 49-50, 129, 175-178, 186, 267, 270, 279-299, 31
0, 346, 367, 381, 384, 468, 472, 475, 478, 481, 528, 562, 620, 636
--------------------------------------------------------------------------
TOTAL                                          208     32    85%

覆盖 ++176 行

Checklist

Add at least a tag in the PR title.
- Tag list: [[FDConfig],[APIServer],[Engine], [Scheduler], [PD Disaggregation], [Executor], [Graph Optimization], [Speculative Decoding], [RL], [Models], [Quantization], [Loader], [OP], [KVCache], [DataProcessor], [BugFix], [Docs], [CI], [Optimization], [Feature], [Benchmark], [Others], [XPU], [HPU], [GCU], [DCU], [Iluvatar], [Metax]]
- You can add new tags based on the PR content, but the semantics must be clear.
Format your code, run pre-commit before commit.
Add unit tests. Please write the reason in this PR if no unit tests.
Provide accuracy results.
If the current PR is submitting to the release branch, make sure the PR has been submitted to the develop branch, then cherry-pick it to the release branch with the [Cherry-Pick] PR tag.

This file contains unit tests for the DeepEP buffer helpers and runners, including various test cases for buffer allocation, cleanup, and dispatching processes.

paddle-bot · 2025-11-15T11:20:40Z

Thanks for your contribution!

Copilot

Pull Request Overview

This PR adds comprehensive unit tests for the DeepEP buffer management and MoE (Mixture of Experts) execution functionality in fastdeploy/model_executor/layers/moe/ep.py, achieving 85% test coverage as part of Hackathon 9th Sprint No.18.

Key Changes:

Added extensive mock infrastructure for testing DeepEP buffers and runners without requiring actual dependencies
Implemented 9 test cases covering buffer allocation, low-latency operations, dispatch/combine flows, and MoE selection strategies
Created recording stubs to verify correct API interactions with the DeepEP communication layer

Copilot · 2025-11-15T11:24:12Z

tests/model_executor/test_ep.py

+        num_max_dispatch_tokens_per_rank=2,
+    )
+    layer = types.SimpleNamespace(
+        redundant_table_manger=None,


Typo in variable name: redundant_table_manger should be redundant_table_manager (missing 'a' in 'manager').

Copilot · 2025-11-15T11:24:12Z

tests/model_executor/test_ep.py

+    TWO_STAGE_RDMA_HINT = 4096
+    TWO_STAGE_NVL_HINT = 2048
+
+    init_history: list[dict] = []


Class-level mutable default list can lead to unexpected behavior. The init_history list is shared across all instances and persists between test runs. While there's a reset() method to clear it, consider using instance-level tracking or document this as an intentional design choice for test recording.

Copilot · 2025-11-15T11:24:12Z

tests/model_executor/test_moe.py

+_STUBS_INSTALLED = False
+
+
+def _install_dependency_stubs():
+    global _STUBS_INSTALLED
+    if _STUBS_INSTALLED:
+        return


Global mutable state _STUBS_INSTALLED can cause test isolation issues. If a test modifies the stubbed modules, subsequent tests may see unexpected state. Consider using pytest fixtures with proper teardown or a context manager to ensure proper isolation between test runs.

Copilot · 2025-11-15T11:24:12Z

tests/model_executor/test_ep.py

+    fd_model_executor_pkg.ops = fd_ops_pkg
+
+    gpu_module = types.ModuleType("fastdeploy.model_executor.ops.gpu")
+    gpu_module.calls = {"redundant": [], "topk": []}


The gpu_module.calls dictionary is initialized with mutable lists that are shared across test runs. While the reset_recorders fixture clears these, consider using a more explicit pattern to avoid potential state leakage if the fixture is accidentally skipped or fails.

Copilot · 2025-11-15T11:24:12Z

tests/model_executor/test_ep.py

+    fd_ops_pkg.gpu = gpu_module
+
+    moe_module = types.ModuleType("fastdeploy.model_executor.layers.moe.moe")
+    moe_module.calls = []


Similar to gpu_module, the moe_module.calls list is a module-level mutable that persists across tests. This pattern is repeated multiple times. Consider creating a shared test utility class to manage these recording structures consistently.

Copilot · 2025-11-15T11:24:13Z

tests/model_executor/test_ep.py

+            return ([0], [0], [1], [2])
+
+    layer = types.SimpleNamespace(
+        redundant_table_manger=_RedundantTable(),


Typo in variable name: redundant_table_manger should be redundant_table_manager (missing 'a' in 'manager').

Copilot · 2025-11-15T11:24:13Z

tests/model_executor/test_ep.py

+        num_max_dispatch_tokens_per_rank=2,
+    )
+    layer = types.SimpleNamespace(
+        redundant_table_manger=None,


Typo in variable name: redundant_table_manger should be redundant_table_manager (missing 'a' in 'manager').

Copilot · 2025-11-15T11:24:13Z

tests/model_executor/test_moe.py

+_STUBS_INSTALLED = False
+
+
+def _install_dependency_stubs():


The _install_dependency_stubs() function is complex and performs significant setup (creating multiple module stubs). Add a docstring explaining its purpose, what dependencies it's stubbing, and why this approach is necessary for testing.

Suggested change

def _install_dependency_stubs():

def _install_dependency_stubs():

"""

Sets up stub modules for external dependencies required by the code under test.

This function creates and registers mock versions of the following modules:

- paddle

- paddle.nn

- paddle.distributed

These stubs provide minimal implementations and attributes needed for the tests to run,

without requiring the actual external libraries to be installed. This approach ensures

that unit tests can execute in isolation, avoiding side effects and dependency issues

that may arise from importing or using real external modules.

"""

Copilot · 2025-11-15T11:24:13Z

tests/model_executor/test_moe.py

+    _STUBS_INSTALLED = True
+
+
+_install_dependency_stubs()


The _install_dependency_stubs() is called at module import time (line 383), which means stubs are installed as soon as the test module is imported. This violates the principle of explicit test setup and can cause side effects. Consider moving this to a pytest fixture with session scope or using pytest's autouse=True fixture for better control and visibility.

Suggested change

_install_dependency_stubs()

@pytest.fixture(scope="session", autouse=True)

def _install_stubs_fixture():

global _STUBS_INSTALLED

if not _STUBS_INSTALLED:

_install_dependency_stubs()

…elpers Refactor DeepEP tests to use scoped stubs

YuanRisheng

把减少的miss代码行数也在pr描述里进行说明

YuanRisheng · 2025-11-19T08:12:54Z

tests/model_executor/test_ep.py

@@ -0,0 +1,631 @@
+"""Unit tests for the DeepEP buffer helpers and runners."""


缺copyright

已经添加，同时已经说明

Added licensing information to the test file.

YuanRisheng · 2025-11-24T07:52:43Z

0.2⭐️

luotao1 · 2025-11-24T07:54:38Z

hi, @xunyoyo

非常感谢你对飞桨的贡献，我们正在运营一个PFCC组织。PFCC是飞桨开源的贡献者俱乐部，只有给飞桨合入过代码的开发者才能加入，俱乐部里每两周会有一次例会（按兴趣参加），也会时不时办线下meetup面基，详情可见 https://github.com/luotao1 主页说明。
如果你对PFCC有兴趣，请发送邮件至 ext_paddle_oss@baidu.com，我们会邀请你加入~

EmmonsCurse · 2025-11-24T08:22:26Z

@xunyoyo @YuanRisheng 该 PR 当前落后 develop 分支约一周的提交。建议后续补充单测类的 PR 在合入前定期执行一次 Update branch，以避免在此期间合入大量更新时产生不必要的冲突。

Add unit tests for DeepEP buffer functionality

4c9ccbe

This file contains unit tests for the DeepEP buffer helpers and runners, including various test cases for buffer allocation, cleanup, and dispatching processes.

Copilot AI review requested due to automatic review settings November 15, 2025 11:20

paddle-bot bot added the contributor External developers label Nov 15, 2025

Copilot started reviewing on behalf of xunyoyo November 15, 2025 11:20 View session

Copilot finished reviewing on behalf of xunyoyo November 15, 2025 11:22

Copilot AI reviewed Nov 15, 2025

View reviewed changes

luotao1 mentioned this pull request Nov 17, 2025

【Hackathon 9th】开源贡献个人挑战赛冲刺赛 PaddlePaddle/Paddle#76333

Open

luotao1 added the PaddlePaddle Hackathon label Nov 17, 2025

luotao1 assigned luotao1 and YuanRisheng Nov 17, 2025

xunyoyo added 2 commits November 17, 2025 21:54

Refactor DeepEP tests to use scoped stubs

2387729

Merge pull request #19 from xunyoyo/codex/add-unit-tests-for-deepep-h…

23813d3

…elpers Refactor DeepEP tests to use scoped stubs

YuanRisheng reviewed Nov 19, 2025

View reviewed changes

Add licensing information to test_ep.py

72bb387

Added licensing information to the test file.

YuanRisheng approved these changes Nov 24, 2025

View reviewed changes

YuanRisheng merged commit 7bac016 into PaddlePaddle:develop Nov 24, 2025
24 of 25 checks passed

-def _install_dependency_stubs():
+def _install_dependency_stubs():
+    """
+    Sets up stub modules for external dependencies required by the code under test.
+    This function creates and registers mock versions of the following modules:
+    - paddle
+    - paddle.nn
+    - paddle.distributed
+    These stubs provide minimal implementations and attributes needed for the tests to run,
+    without requiring the actual external libraries to be installed. This approach ensures
+    that unit tests can execute in isolation, avoiding side effects and dependency issues
+    that may arise from importing or using real external modules.
+    """

		@@ -0,0 +1,631 @@
		"""Unit tests for the DeepEP buffer helpers and runners."""

[CI] 【Hackathon 9th Sprint No.18】NO.18 功能模块单测补充 #5064

[CI] 【Hackathon 9th Sprint No.18】NO.18 功能模块单测补充 #5064

Uh oh!

Conversation

xunyoyo commented Nov 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Modifications

Usage or Command

Accuracy Tests

Checklist

Uh oh!

paddle-bot bot commented Nov 15, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Copilot AI Nov 15, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 15, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 15, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 15, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 15, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 15, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 15, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 15, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Nov 15, 2025

Choose a reason for hiding this comment

Uh oh!

YuanRisheng left a comment

Choose a reason for hiding this comment

Uh oh!

YuanRisheng Nov 19, 2025

Choose a reason for hiding this comment

Uh oh!

xunyoyo Nov 19, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

YuanRisheng commented Nov 24, 2025

Uh oh!

luotao1 commented Nov 24, 2025

Uh oh!

EmmonsCurse commented Nov 24, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

xunyoyo commented Nov 15, 2025 •

edited

Loading