[FEAT][executors]: optimize advanced vLLM sampler with shared prefix caching #11
Description
Metadata
Metadata
Assignees
Labels
Tasks involving the interaction of vLLM inference and DeepSpeed training endpoints.New feature or requestSpecific optimizations or bugs in NVIDIA graphics cards (such as FlashInfer, TMA optimizations)Specific tasks specific to AMD graphics cards (such as CK, bpreshuffle/FA)Performance optimization tasks aimed at increasing throughput and reducing latency etc.
Type
Fields
Give feedbackNo fields configured for issues without a type.
Refactor the rollout sampler to explicitly enable vLLM's Shared Prefix Caching mechanism. This is critical to eliminate KV cache redundancy during multi-candidate generation per prompt in GRPO.