[Feature] Unify cache aware routing for PD/DP routers

[simondanielsson]

Purpose

PD router now derives routing text from the typed request using the same logic as DP.

Previously, the PD router previously dumped the entire request JSON into text and used that for routing. The DP router instead used extract_text_for_routing which correctly converts a request into text based on the type of request and endpoint (and handles cases when e.g. raw token ids are passed or when we need to route only using the session-id). This PR now changes so the PD router also uses extract_text_for_routing.

Examples:

1. a ChatCompletionRequest routes based on session id, so the relevant text to use for routing is the session id. Hence extract_text_for_routing gives the session id
2. a CompletionRequest routes based on prompt content. Hence, the extract_text_for_routing returns the prompt text and nothing else.

We subsequently pass this type-specific request text to the routing policy at hand.

Test Plan

New tests pass

Test Result

OK

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bb54065a9d

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[Copilot]

🟡 Changes recommended

The new routing_text helper drops empty extracted routing text to None, which can change policy behavior compared to the regular router and should be aligned before merging.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

This PR updates the vLLM PD router to derive cache-aware / consistent-hash routing text from typed requests (via extract_text_for_routing) to match the behavior of the regular (DP) router, instead of hashing the full serialized request JSON.

Changes:

• Added helpers to determine whether configured policies require request text and to compute routing text from GenerationRequest::extract_text_for_routing.
• Threaded an optional request_text through PD routing paths (process_vllm_request / process_transparent) to avoid re-serializing full request bodies.
• Added unit tests validating typed routing text behavior and preserving legacy serialization behavior for untyped transparent routing.

File summaries

File	Description
src/routers/http/vllm_pd_router.rs	Computes PD routing text from typed requests (GenerationRequest) and passes it into PD policy selection, with tests for typed vs transparent routing text behavior.

Review details

• Files reviewed: 1/1 changed files
• Comments generated: 2
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.