[Docs] Update KubeRay Serve High Throughput guide to Ray 2.56 - #64144
Conversation
Update the document to use Ray 2.56 as prerequisite and add new environment variables: - RAY_SERVE_LLM_ENABLE_DIRECT_STREAMING - VLLM_USE_RAY_V2_EXECUTOR_BACKEND Signed-off-by: Spencer Peterson <spencerjp@google.com>
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request updates the KubeRay high-throughput documentation for Ray 2.56, introducing new environment variables (RAY_SERVE_LLM_ENABLE_DIRECT_STREAMING and VLLM_USE_RAY_V2_EXECUTOR_BACKEND) and adding links to sample configurations. The review feedback highlights two issues: a relative link to a sample YAML file that will break on the documentation website, and a bash syntax error where combining variable assignment and command execution on a single line prevents proper variable expansion.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
Signed-off-by: Spencer Peterson <spencerjp@google.com>
ray-gardener
Bot
added
serve
…oject#64144) ## Description Update the document to use Ray 2.56 as prerequisite and add new environment variables: - RAY_SERVE_LLM_ENABLE_DIRECT_STREAMING - VLLM_USE_RAY_V2_EXECUTOR_BACKEND ## Related issues N/A --------- Signed-off-by: Spencer Peterson <spencerjp@google.com>
4 participants
Description
Update the document to use Ray 2.56 as prerequisite and add new environment variables:
Related issues
N/A