Skip to content

Comments

[Test] UT for linear method patch#281

Closed
rjg-lyh wants to merge 1 commit intovllm-project:v0.7.3-devfrom
rjg-lyh:pr-ut-linear
Closed

[Test] UT for linear method patch#281
rjg-lyh wants to merge 1 commit intovllm-project:v0.7.3-devfrom
rjg-lyh:pr-ut-linear

Conversation

@rjg-lyh
Copy link
Collaborator

@rjg-lyh rjg-lyh commented Mar 9, 2025

What this PR does / why we need it?

UT for linear method patch.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

None.

Signed-off-by: rjg-lyh <1318825571@qq.com>
@MengqingCao
Copy link
Collaborator

This relys on multi-npu ci machine, cc @Potabk
@rjg-lyh Please firstly make CI happy with code format check, thx!

@Potabk
Copy link
Collaborator

Potabk commented Mar 27, 2025

This relys on multi-npu ci machine, cc @Potabk @rjg-lyh Please firstly make CI happy with code format check, thx!

Currently, all our unit tests are run on a single NPU. and we now have multiple NPU runners but cannot flexibly allocate them according to test requirements. here is a solution: mark a test as single or multiple to adapt to the runner's filtering, just like the example following:

@pytest.mark.multi
@pytest.mark.parametrize(
    "model, distributed_executor_backend", [
        ("Qwen/QwQ-32B", "mp"),
    ])
def test_models_distributed(
    vllm_runner,
    example_prompts,
    model: str,
    distributed_executor_backend: str,
) -> None:
    dtype = "half"
    max_tokens = 5
    with vllm_runner(model,
                     dtype=dtype,
                     tensor_parallel_size=2,
                     distributed_executor_backend=distributed_executor_backend
                     ) as vllm_model:
        vllm_model.generate_greedy(example_prompts, max_tokens)

and then we will run the multi-npu test case with command pytest -sv -m multi test/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants