400 err with vLLM Qwen3.8 27B #4937

EthanPeng-2048 · 2026-08-28T16:27:13Z

EthanPeng-2048
Aug 28, 2026

本轮运行失败400: {"message":"Unexpected message role.","type":"BadRequestError","param":null,"code":400}

(APIServer pid=1272) INFO: 127.0.0.1:59873 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request
(APIServer pid=1272) ERROR: Exception in ASGI application
(APIServer pid=1272) Traceback (most recent call last):
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/vllm/renderers/hf.py", line 672, in safe_apply_chat_template
(APIServer pid=1272) return tokenizer.apply_chat_template(
(APIServer pid=1272) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/vllm/tokenizers/hf.py", line 62, in apply_chat_template
(APIServer pid=1272) return tok.apply_chat_template(*args, **kwargs)
(APIServer pid=1272) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/transformers/tokenization_utils_base.py", line 3108, in apply_chat_template
(APIServer pid=1272) rendered_chat, generation_indices = render_jinja_template(
(APIServer pid=1272) ^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/transformers/utils/chat_template_utils.py", line 581, in render_jinja_template
(APIServer pid=1272) rendered_chat = compiled_template.render(
(APIServer pid=1272) ^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/jinja2/environment.py", line 1295, in render
(APIServer pid=1272) self.environment.handle_exception()
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/jinja2/environment.py", line 942, in handle_exception
(APIServer pid=1272) raise rewrite_traceback_stack(source=source)
(APIServer pid=1272) File "", line 160, in top-level template code
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/jinja2/sandbox.py", line 401, in call
(APIServer pid=1272) return __context.call(__obj, *args, **kwargs)
(APIServer pid=1272) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/transformers/utils/chat_template_utils.py", line 479, in raise_exception
(APIServer pid=1272) raise jinja2.exceptions.TemplateError(message)
(APIServer pid=1272) jinja2.exceptions.TemplateError: Unexpected message role.
(APIServer pid=1272)
(APIServer pid=1272) The above exception was the direct cause of the following exception:
(APIServer pid=1272)
(APIServer pid=1272) Traceback (most recent call last):
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/uvicorn/protocols/http/httptools_impl.py", line 422, in run_asgi
(APIServer pid=1272) result = await app( # type: ignore[func-returns-value]
(APIServer pid=1272) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", line 63, in call
(APIServer pid=1272) return await self.app(scope, receive, send)
(APIServer pid=1272) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/fastapi/applications.py", line 1159, in call
(APIServer pid=1272) await super().call(scope, receive, send)
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/starlette/applications.py", line 107, in call
(APIServer pid=1272) await self.middleware_stack(scope, receive, send)
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 186, in call
(APIServer pid=1272) raise exc
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/starlette/middleware/errors.py", line 164, in call
(APIServer pid=1272) await self.app(scope, receive, _send)
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/starlette/middleware/cors.py", line 87, in call
(APIServer pid=1272) await self.app(scope, receive, send)
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/prometheus_fastapi_instrumentator/middleware.py", line 177, in call
(APIServer pid=1272) raise exc
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/prometheus_fastapi_instrumentator/middleware.py", line 175, in call
(APIServer pid=1272) await self.app(scope, receive, send_wrapper)
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/starlette/middleware/exceptions.py", line 63, in call
(APIServer pid=1272) await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
(APIServer pid=1272) raise exc
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
(APIServer pid=1272) await app(scope, receive, sender)
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in call
(APIServer pid=1272) await self.app(scope, receive, send)
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/starlette/routing.py", line 716, in call
(APIServer pid=1272) await self.middleware_stack(scope, receive, send)
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/starlette/routing.py", line 736, in app
(APIServer pid=1272) await route.handle(scope, receive, send)
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/starlette/routing.py", line 290, in handle
(APIServer pid=1272) await self.app(scope, receive, send)
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 134, in app
(APIServer pid=1272) await wrap_app_handling_exceptions(app, request)(scope, receive, send)
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
(APIServer pid=1272) raise exc
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
(APIServer pid=1272) await app(scope, receive, sender)
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 120, in app
(APIServer pid=1272) response = await f(request)
(APIServer pid=1272) ^^^^^^^^^^^^^^^^
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 674, in app
(APIServer pid=1272) raw_response = await run_endpoint_function(
(APIServer pid=1272) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/fastapi/routing.py", line 328, in run_endpoint_function
(APIServer pid=1272) return await dependant.call(**values)
(APIServer pid=1272) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/vllm/entrypoints/utils.py", line 96, in wrapper
(APIServer pid=1272) return handler_task.result()
(APIServer pid=1272) ^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/vllm/entrypoints/utils.py", line 117, in wrapper
(APIServer pid=1272) return await func(*args, **kwargs)
(APIServer pid=1272) ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/api_router.py", line 61, in create_chat_completion
(APIServer pid=1272) generator = await handler.create_chat_completion(request, raw_request)
(APIServer pid=1272) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 240, in create_chat_completion
(APIServer pid=1272) return await self._with_kv_transfer_rejection_cleanup(
(APIServer pid=1272) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/vllm/entrypoints/openai/engine/serving.py", line 431, in _with_kv_transfer_rejection_cleanup
(APIServer pid=1272) return await awaitable
(APIServer pid=1272) ^^^^^^^^^^^^^^^
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 259, in _create_chat_completion
(APIServer pid=1272) result = await self.render_chat_request(request)
(APIServer pid=1272) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/vllm/entrypoints/openai/chat_completion/serving.py", line 226, in render_chat_request
(APIServer pid=1272) return await self.openai_serving_render.render_chat(request)
(APIServer pid=1272) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/vllm/entrypoints/serve/render/serving.py", line 250, in render_chat
(APIServer pid=1272) conversation, engine_inputs = await self.preprocess_chat(
(APIServer pid=1272) ^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/vllm/entrypoints/serve/render/serving.py", line 562, in preprocess_chat
(APIServer pid=1272) (conversation,), (engine_input,) = await renderer.render_chat_async(
(APIServer pid=1272) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/vllm/renderers/base.py", line 1019, in render_chat_async
(APIServer pid=1272) for conv, prompt in await asyncio.gather(*rendered):
(APIServer pid=1272) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/vllm/renderers/hf.py", line 968, in render_messages_async
(APIServer pid=1272) prompt_raw = await self._apply_chat_template_async(
(APIServer pid=1272) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1272) File "/home/ethan/.pyenv/versions/3.12.13/lib/python3.12/concurrent/futures/thread.py", line 59, in run
(APIServer pid=1272) result = self.fn(*self.args, **self.kwargs)
(APIServer pid=1272) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(APIServer pid=1272) File "/home/ethan/codes/1cat-vllm/.venv/lib/python3.12/site-packages/vllm/renderers/hf.py", line 687, in safe_apply_chat_template
(APIServer pid=1272) raise ValueError(str(e)) from e
(APIServer pid=1272) ValueError: Unexpected message role.

173787247 · 2026-08-29T13:56:03Z

173787247
Aug 29, 2026

更正：上一条完全偏题，跟 sandbox 无关。请看楼下 @EthanPeng-2048 的说明——根因是消息角色 developer；vLLM + Qwen chat template 通常只接受 system / user / assistant（以及部分工具角色），遇到 developer 就会在 apply_chat_template 里抛 Unexpected message role 并返回 400。

抱歉刷了无关模板。

1 reply

EthanPeng-2048 Aug 30, 2026
Author

沙箱 / 权限（「400 err with vLLM Qwen3.8 27B」）

确认当前 session 的 sandbox 模式（workspace-write / danger-full-access 等）

非扩权请求若被实现成抛异常，看起来会像「所有工具都挂」——升级到含修复的版本，或先在受控环境用更宽权限验证是否为该 bug

改权限相关配置后重启并新开会话

把 dsh 精确版本号和触发工具名贴出来，便于对照 changelog。

问题在于developer角色，vLLM不接受这个角色，只接受system

173787247 · 2026-08-30T13:09:43Z

173787247
Aug 30, 2026

你说得对，根因是角色，不是 sandbox。

vLLM 走 HuggingFace apply_chat_template 时，Qwen 模板一般只认 system / user / assistant（以及部分工具相关角色）。DSH / 某些 OpenAI 兼容路径会把「开发者说明」发成 developer；模板里没有这个角色，就会在 Jinja 里抛 Unexpected message role，上游直接 400。

临时绕过：

看实际请求体里有没有 "role":"developer"（代理录一份，或开 vLLM 请求日志）。
若有，在进 vLLM 前把它改成 system（多条 system 通常能拼），或换一个接受 developer 的服务端/模板。
新开会话后再试；旧轨迹里若已混进非法角色，可能继续炸。

产品侧更干净的修法：对「非官方 DeepSeek、走 HF chat template 的本地后端」做 role 归一化（developer → system），而不是原样转发。

0 replies

PerryLink · 2026-08-30T14:34:24Z

PerryLink
Aug 30, 2026

注：本条评论原文因编码问题曾显示为乱码，已重建为可读版（2026-09-07）。

这个报错（模型 400 错误）多半是 developer 消息在 vLLM + Qwen chat template 上的兼容问题，两层看：

vLLM 侧：确认 developer 角色映射是否正确，必要时显式传 --chat-template；
DSH 侧：把 provider 配置里「思考模式」与 developer 消息的映射对齐——两者不一致会触发 400。

另外推荐：用 dsh-translate 做供应商参数翻译，把模型间参数自动对齐，可以绕开这类错误，欢迎试用。

0 replies

weijiafu14 · 2026-08-30T14:39:33Z

weijiafu14
Aug 30, 2026

楼上把根因（developer 角色撞 Qwen chat template）定得很准。补上已验证的配置修法——这正是我们录过请求体断言过的那个 compat 开关：

给这条 vLLM 路由的模型声明加 compat: { supportsDeveloperRole: false }——系统提示词就会以 system 角色发出，Qwen 模板照单全收。这不是文档抄的：我们的验证装置用透传录制代理抓真实请求体，断言的就是"声明了 supportsDeveloperRole: false 之后请求里不再出现 developer 角色"（examples/gateway-compat，那条断言曾专门为防"默认值假绿"重写过）。声明整体写法：examples/custom-gateways。

另外这个坑的系统性讨论在 #3789（提议非原生 OpenAI 端点默认 supportsDeveloperRole: false + 配置 UI 显式开关）——vLLM/Qwen 一族撞得足够频繁，值得去那边 +1。

0 replies

Navigation Menu

400 err with vLLM Qwen3.8 27B #4937

Uh oh!

{{title}}

Uh oh!

Replies: 4 comments 1 reply

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Uh oh!

{{title}}

Uh oh!

沙箱 / 权限（「400 err with vLLM Qwen3.8 27B」）

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

400 err with vLLM Qwen3.8 27B #4937

Uh oh!

EthanPeng-2048 Aug 28, 2026

Replies: 4 comments · 1 reply

Uh oh!

Uh oh!

173787247 Aug 29, 2026

Uh oh!

EthanPeng-2048 Aug 30, 2026 Author

沙箱 / 权限（「400 err with vLLM Qwen3.8 27B」）

Uh oh!

173787247 Aug 30, 2026

Uh oh!

Uh oh!

PerryLink Aug 30, 2026

Uh oh!

weijiafu14 Aug 30, 2026

EthanPeng-2048
Aug 28, 2026

Replies: 4 comments 1 reply

173787247
Aug 29, 2026

EthanPeng-2048 Aug 30, 2026
Author

173787247
Aug 30, 2026

PerryLink
Aug 30, 2026

weijiafu14
Aug 30, 2026