[Serve][LLM] Replace SGLang format_messages_to_prompt with _build_chat_messages

[eicherseiji]

Fix

[2026-02-26T22:03:30Z] use logger.warning(......................................................Passed
[2026-02-26T22:03:30Z] + for HOOK in "${HOOKS[@]}"
[2026-02-26T22:03:30Z] + pre-commit run ruff --all-files --show-diff-on-failure
[2026-02-26T22:03:30Z] [INFO] Installing environment for https://github.com/astral-sh/ruff-pre-commit.
[2026-02-26T22:03:30Z] [INFO] Once installed this environment will be reused.
[2026-02-26T22:03:30Z] [INFO] This may take a few minutes...
[2026-02-26T22:03:41Z] ruff.....................................................................Failed
[2026-02-26T22:03:41Z] - hook id: ruff
[2026-02-26T22:03:41Z] - exit code: 1
[2026-02-26T22:03:41Z]
[2026-02-26T22:03:41Z] python/ray/llm/examples/sglang/modules/sglang_engine.py:330:22: F821 Undefined name `format_messages_to_prompt`
[2026-02-26T22:03:41Z]     |
[2026-02-26T22:03:41Z] 328 |         else:
[2026-02-26T22:03:41Z] 329 |             # Chat embedding request - convert messages to prompt
[2026-02-26T22:03:41Z] 330 |             prompt = format_messages_to_prompt(request.messages)
[2026-02-26T22:03:41Z]     |                      ^^^^^^^^^^^^^^^^^^^^^^^^^ F821
[2026-02-26T22:03:41Z] 331 |
[2026-02-26T22:03:41Z] 332 |         # async_encode handles both single strings and lists of strings
[2026-02-26T22:03:41Z]     |
[2026-02-26T22:03:41Z]
[2026-02-26T22:03:41Z] Found 1 error.
[2026-02-26T22:03:41Z] All checks passed!
[2026-02-26T22:03:41Z]
[2026-02-26T22:03:42Z] ruff.....................................................................Passed
[2026-02-26T22:03:42Z] 🚨 Error: The command exited with status 1

[gemini-code-assist]

Code Review

This pull request fixes a crash in the SGLang example caused by an undefined function call. The fix correctly replaces the call with existing helper methods from the same class. My review includes one suggestion to adjust the prompt formatting for embedding requests to ensure the resulting embedding is not unintentionally skewed by a generation-specific token.

[gemini-code-assist]

The _render_fallback_prompt method appends assistant: to the prompt, which is suitable for generation tasks to prompt the model for a response. However, for embedding requests, the goal is typically to get a representation of the existing conversation history. Including this generation prompt might skew the resulting embedding vector.

To ensure the embedding accurately reflects only the provided conversation, consider creating the prompt from the chat messages directly, without appending the assistant: token.

Suggested change

	prompt = self._render_fallback_prompt(
	self._build_chat_messages(request.messages)
	)
	chat_messages = self._build_chat_messages(request.messages)
	prompt_lines = []
	for message in chat_messages:
	role = str(message.get("role", "user"))
	content = message.get("content", "") or ""
	prompt_lines.append(f"{role}: {content}")
	prompt = "\n".join(prompt_lines)

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

[cursor]

Fallback prompt appends "assistant:" unsuitable for embeddings

Medium Severity

_render_fallback_prompt unconditionally appends "assistant:" at the end of the rendered prompt (line 153), which is designed for text generation to cue the model to produce the next assistant turn. For embedding requests, this trailing "assistant:" contaminates the text being encoded, producing embeddings that don't faithfully represent the actual conversation content. The embedding code path now always uses this generation-oriented renderer rather than a plain message concatenation.

Additional Locations (1)

• python/ray/llm/examples/sglang/modules/sglang_engine.py#L142-L154