Repository navigation
v0.7.159
vLLM tier priority (experiential#1214): GatewayRungDispatchPolicy.upstream_priority (default off; adds no identity bytes when unset). On an opted-in rung using the openai_compatible or openai_responses wire, each dispatch sends vLLM's priority body field from the caller's admission level: Pro 0, paying 1, free 2. A vLLM server run with --scheduling-policy priority then queues requests by tier, and when it must preempt, picks free first. Rungs that have not opted in never receive the field. Native 0.3.133 (unchanged).