[Bug] [Bug] sglang.bench_serving backend vllm error

[V-yw]

Checklist

• 1. I have searched related issues but cannot get the expected help.
• 2. The bug has not been fixed in the latest version.
• 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.
• 4. If the issue you raised is not a bug but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
• 5. Please use English, otherwise it will be closed.

Describe the bug

try:
(sglang) root@megaium-H8230:~# python -m sglang.bench_serving --backend vllm --base-url http://localhost:6006/ --num-prompts 100 --max-concurrency 4 --model /opt/deepseek-r1-70b-q4 --dataset-path /root/ShareGPT_V3_unfiltered_cleaned_split.json

Namespace(backend='vllm', base_url='http://localhost:6006/', host='0.0.0.0', port=8000, dataset_name='sharegpt', dataset_path='/root/ShareGPT_V3_unfiltered_cleaned_split.json', model='/opt/deepseek-r1-70b-q4', tokenizer=None, num_prompts=100, sharegpt_output_len=None, sharegpt_context_len=None, random_input_len=1024, random_output_len=1024, random_range_ratio=0.0, request_rate=inf, max_concurrency=4, multi=False, request_rate_range='2,34,2', output_file=None, disable_tqdm=False, disable_stream=False, return_logprob=False, seed=1, disable_ignore_eos=False, extra_request_body=None, apply_chat_template=False, profile=False, lora_name=None, gsp_num_groups=64, gsp_prompts_per_group=16, gsp_system_prompt_len=2048, gsp_question_len=128, gsp_output_len=256)

#Input tokens: 33559
#Output tokens: 21265
Starting initial single prompt test run...
Traceback (most recent call last):
File "/root/miniconda3/envs/sglang/lib/python3.10/runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/root/miniconda3/envs/sglang/lib/python3.10/runpy.py", line 86, in _run_code
exec(code, run_globals)
File "/root/miniconda3/envs/sglang/lib/python3.10/site-packages/sglang/bench_serving.py", line 1513, in
run_benchmark(args)
File "/root/miniconda3/envs/sglang/lib/python3.10/site-packages/sglang/bench_serving.py", line 1273, in run_benchmark
return asyncio.run(
File "/root/miniconda3/envs/sglang/lib/python3.10/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/root/miniconda3/envs/sglang/lib/python3.10/asyncio/base_events.py", line 641, in run_until_complete
return future.result()
File "/root/miniconda3/envs/sglang/lib/python3.10/site-packages/sglang/bench_serving.py", line 944, in benchmark
raise ValueError(
ValueError: Initial test run failed - Please make sure benchmark arguments are correctly specified. Error: Not Found

Reproduction

server:
python -m vllm.entrypoints.openai.api_server --model /opt/deepseek-r1-70b-q4 --served-model-name deepseek-r1-70b-q4 --tensor-parallel-size 1 --pipeline-parallel-size 1 --block-size 64 --port 6006 --max_num_seqs 256 --trust-remote-code --max-model-len 8192

client:
python -m sglang.bench_serving --backend vllm --base-url http://localhost:6006/ --num-prompts 100 --max-concurrency 4 --model /opt/deepseek-r1-70b-q4 --dataset-path /root/ShareGPT_V3_unfiltered_cleaned_split.json

Environment

sglang 0.4.3.post2

[V-yw]

@mutinifni Could you help me find out what the reason is?

[V-yw]

result error:
INFO: 127.0.0.1:45492 - "POST /v1/completions HTTP/1.1" 404 Not Found

[OctPeng]

I have encountered the same problem, but I haven't solved it yet.

[AndrewTsao]

尝试一下这类参数：

python -msglang.bench_serving --backend vllm --host hpc-deep01 --port 1234 --dataset-name sharegpt --num-prompts 10  --model DeepSeek-R1 --max-concurrency 1

在当前目录造下 DeepSeek-R1 模型的符号链接，模型的路径必须与vllm的一致。如果是 deepseek-ai/DeepSeek-R1，此处也应该这样。然后，目前还有个问题，对OPEN_AI_KEY的参数支持，参考这个： #3870

[github-actions]

This issue has been automatically closed due to inactivity. Please feel free to reopen it if needed.