Several minor tweak to vllm benchmark workflow #114

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Merged

huydhn merged 3 commits into main from vllm-benchmark-tweaks-1213

Dec 15, 2025

Contributor

huydhn commented Dec 13, 2025

Several small tweaks:

Disable arm benchmark for now as it's not working and there is no owner
Only need to parse the list of model from serving benchmark config. It always has all the models because serving benchmark is the most basic one, and it has tensor_parallel_size field set correctly. This is to fix a non deterministic issue where a model's tensor_parallel_size could be missing when glob decides to return the list of benchmark configs differently
Run daily, saving the capacity for the upcoming nightly run

huydhn requested a review from yangw-dev

December 13, 2025 08:32

meta-cla bot added the cla signed label

huydhn had a problem deploying to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Error

huydhn temporarily deployed to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Inactive

huydhn temporarily deployed to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Inactive

huydhn temporarily deployed to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Inactive

huydhn had a problem deploying to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Error

huydhn temporarily deployed to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Inactive

huydhn had a problem deploying to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Error

huydhn had a problem deploying to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Error

huydhn temporarily deployed to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Inactive

huydhn had a problem deploying to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Error

huydhn had a problem deploying to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Error

huydhn temporarily deployed to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Inactive

huydhn temporarily deployed to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Inactive

huydhn temporarily deployed to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Inactive

huydhn had a problem deploying to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Error

huydhn had a problem deploying to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Error

huydhn temporarily deployed to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Inactive

huydhn temporarily deployed to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Inactive

huydhn temporarily deployed to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Inactive

huydhn temporarily deployed to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Inactive

huydhn had a problem deploying to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Error

huydhn had a problem deploying to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Error

huydhn had a problem deploying to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Error

huydhn had a problem deploying to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Error

huydhn had a problem deploying to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Error

huydhn temporarily deployed to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Inactive

huydhn had a problem deploying to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Error

huydhn temporarily deployed to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Inactive

huydhn temporarily deployed to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Inactive

huydhn temporarily deployed to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Inactive

huydhn had a problem deploying to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Error

huydhn had a problem deploying to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Error

huydhn temporarily deployed to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Inactive

huydhn had a problem deploying to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Error

huydhn had a problem deploying to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Error

huydhn had a problem deploying to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Error

huydhn temporarily deployed to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Inactive

huydhn temporarily deployed to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Inactive

huydhn temporarily deployed to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Inactive

huydhn temporarily deployed to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Inactive

huydhn had a problem deploying to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Failure

huydhn had a problem deploying to pytorch-x-vllm

December 13, 2025 08:32

— with

GitHub Actions Error


          [no ci] Several minor tweak to vllm benchmark workflow

c4dcf9f

Signed-off-by: Huy Do <huydhn@gmail.com>

huydhn force-pushed the vllm-benchmark-tweaks-1213 branch from 043f5b1 to c4dcf9f Compare

December 13, 2025 08:34

huydhn added 2 commits

December 13, 2025 00:45


          [no ci] Several minor tweak to vllm benchmark workflow

7ba6367

Signed-off-by: Huy Do <huydhn@gmail.com>


          [no ci] Merge branch 'main' into vllm-benchmark-tweaks-1213

e5d0271

Signed-off-by: Huy Do <huydhn@gmail.com>

yangw-dev approved these changes

View reviewed changes

huydhn merged commit c4ccc4d into main

1 check passed

fadara01 added a commit that referenced this pull request


          Re-enable Arm CPU vLLM HUD Benchmarks

455c0b3

- re-enables the benchmarks disabled in #114
- related to vllm-project/vllm#26494 (not sure which needs to go in first)

Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>

fadara01 mentioned this pull request

Re-enable Arm CPU vLLM HUD Benchmarks #120

Open

fadara01 added a commit that referenced this pull request


          Re-enable Arm CPU vLLM HUD Benchmarks

04a5443

- re-enables the benchmarks disabled in #114
- related to vllm-project/vllm#26494 (not sure which needs to go in first)
- use default block_size in serving benchmarks (i.e. 128 instead of setting it to 16)

Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>

fadara01 added a commit that referenced this pull request


          Re-enable Arm CPU vLLM HUD Benchmarks

beaad7c

- re-enables the benchmarks disabled in #114
- related to vllm-project/vllm#26494 (not sure which needs to go in first)
- use default block_size in serving benchmarks (i.e. 128 instead of setting it to 16)

Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>

fadara01 added a commit that referenced this pull request


          Re-enable Arm CPU vLLM HUD Benchmarks

36d2456

- re-enables the benchmarks disabled in #114
- related to vllm-project/vllm#26494 (not sure which needs to go in first)
- use default block_size in serving benchmarks (i.e. 128 instead of setting it to 16)

Signed-off-by: Fadi Arafeh <fadi.arafeh@arm.com>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels