not for land: vllm bench for mxfp8

[vkuzo]

Summary:

with-proxy python
benchmarks/mx_formats/vllm/create_quantized_hf_model.py
~/local/tmp/20251203_test_model_mxfp8

with-proxy vllm bench throughput --model
~/local/tmp/20251203_test_model_mxfp8/ --dataset-name sonnet
--dataset-path ~/local/vllm/benchmarks/sonnet.txt --num-prompts 1024
--tensor-parallel-size 1 --max-model-len 2048 --gpu-memory-utilization
0.8

currently fails with compile error (PyTorch 2.9)

Test Plan:

Reviewers:

Subscribers:

Tasks:

Tags:

[vkuzo]

Stack from ghstack (oldest at bottom):

• -> not for land: vllm bench for mxfp8 #3426

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/3426

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 2 Unrelated Failures

As of commit ec96c24 with merge base 16aad7c ():

NEW FAILURE - The following job has failed:

• PR Label Check / Check PR Labels (gh)
  Process completed with exit code 1.

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Run Regression Tests (aarch64) / test-cpu-ops (linux.arm64.2xlarge) (gh) (trunk failure)
  ##[error]The operation was canceled.
• Run Regression Tests (aarch64) / test-cpu-ops (macos-14) (gh) (trunk failure)
  Process completed with exit code 139.

This comment was automatically generated by Dr. CI and updates every 15 minutes.