[serve] Track actor fallback_strategy on DeploymentSchedulingInfo#64346
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces tracking for the fallback_strategy in DeploymentSchedulingInfo and populates it from replica_config.ray_actor_options when a deployment is deployed. The reviewer suggests leveraging this newly tracked fallback_strategy to improve the downscaling logic in _get_replicas_to_stop by prioritizing stopping replicas on lower-priority fallback nodes.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| label_selector: Optional[Dict[str, str]] = None | ||
| placement_group_bundles: Optional[List[RequestedResources]] = None | ||
| bundle_label_selector: Optional[List[Dict[str, str]]] = None | ||
| fallback_strategy: Optional[List[Dict[str, Any]]] = None |
There was a problem hiding this comment.
Now that fallback_strategy is tracked on DeploymentSchedulingInfo, we have an opportunity to improve the downscaling logic in _get_replicas_to_stop. Currently, _get_replicas_to_stop treats all replicas on fallback nodes equally (as non-matching). By leveraging the tracked fallback_strategy, we can prioritize stopping replicas on lower-priority fallback nodes (or default nodes) over higher-priority fallback nodes during downscaling.
DeploymentSchedulingInfo records the actor label_selector and the placement-group bundle_label_selector from on_deployment_deployed, but not the actor fallback_strategy. Add it for symmetry so a scheduler that consults the per-deployment DeploymentSchedulingInfo (rather than the per-replica ReplicaSchedulingRequest) has access to the fallback strategy. Additive only; no behavior change in the default deployment scheduler. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: john.taylor <john.taylor@anyscale.com>
Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: john.taylor <john.taylor@anyscale.com>
515ce27 to
df24278
Compare
…y-project#64346) ## Why are these changes needed? `DeploymentScheduler.on_deployment_deployed` records a deployment's scheduling constraints on `DeploymentSchedulingInfo` — including the actor `label_selector` and the placement-group `bundle_label_selector` — but it does not record the actor `fallback_strategy`. This adds a `fallback_strategy` field to `DeploymentSchedulingInfo` and populates it from `replica_config.ray_actor_options`, for parity with the other recorded scheduling options. This makes the actor fallback strategy available to schedulers that reason over the per-deployment `DeploymentSchedulingInfo` rather than the per-replica `ReplicaSchedulingRequest`. The change is purely additive — one optional dataclass field plus its assignment in `on_deployment_deployed`. There is **no behavior change** in `DefaultDeploymentScheduler`, which builds placement candidates from the `ReplicaSchedulingRequest` (already carrying `fallback_strategy` in `actor_options`). ```python # DeploymentSchedulingInfo bundle_label_selector: Optional[List[Dict[str, str]]] = None + fallback_strategy: Optional[List[Dict[str, Any]]] = None placement_group_strategy: Optional[str] = None # DeploymentScheduler.on_deployment_deployed info.bundle_label_selector = replica_config.placement_group_bundle_label_selector + info.fallback_strategy = replica_config.ray_actor_options.get("fallback_strategy") info.max_replicas_per_node = replica_config.max_replicas_per_node ``` ## Related issue number N/A ## Checks - [x] I've signed off every commit (`git commit -s`) in this PR. - [ ] I've run `scripts/format.sh` to lint the changes in this PR. - [ ] I've included any doc changes needed — N/A (internal field, no API surface). - [ ] I've made sure the tests are passing. - [ ] Testing Strategy - [x] Unit tests - [ ] This PR is not tested :( --------- Signed-off-by: john.taylor <john.taylor@anyscale.com> Co-authored-by: Claude <noreply@anthropic.com>
Why are these changes needed?
DeploymentScheduler.on_deployment_deployedrecords a deployment's scheduling constraints onDeploymentSchedulingInfo— including the actorlabel_selectorand the placement-groupbundle_label_selector— but it does not record the actorfallback_strategy. This adds afallback_strategyfield toDeploymentSchedulingInfoand populates it fromreplica_config.ray_actor_options, for parity with the other recorded scheduling options.This makes the actor fallback strategy available to schedulers that reason over the per-deployment
DeploymentSchedulingInforather than the per-replicaReplicaSchedulingRequest. The change is purely additive — one optional dataclass field plus its assignment inon_deployment_deployed. There is no behavior change inDefaultDeploymentScheduler, which builds placement candidates from theReplicaSchedulingRequest(already carryingfallback_strategyinactor_options).Related issue number
N/A
Checks
git commit -s) in this PR.scripts/format.shto lint the changes in this PR.