Skip to content

[serve] Track actor fallback_strategy on DeploymentSchedulingInfo#64346

Merged
abrarsheikh merged 2 commits into
ray-project:masterfrom
johntaylor-cell:serve-fallback-strategy-on-scheduling-info
Jun 26, 2026
Merged

[serve] Track actor fallback_strategy on DeploymentSchedulingInfo#64346
abrarsheikh merged 2 commits into
ray-project:masterfrom
johntaylor-cell:serve-fallback-strategy-on-scheduling-info

Conversation

@johntaylor-cell

Copy link
Copy Markdown
Contributor

Why are these changes needed?

DeploymentScheduler.on_deployment_deployed records a deployment's scheduling constraints on DeploymentSchedulingInfo — including the actor label_selector and the placement-group bundle_label_selector — but it does not record the actor fallback_strategy. This adds a fallback_strategy field to DeploymentSchedulingInfo and populates it from replica_config.ray_actor_options, for parity with the other recorded scheduling options.

This makes the actor fallback strategy available to schedulers that reason over the per-deployment DeploymentSchedulingInfo rather than the per-replica ReplicaSchedulingRequest. The change is purely additive — one optional dataclass field plus its assignment in on_deployment_deployed. There is no behavior change in DefaultDeploymentScheduler, which builds placement candidates from the ReplicaSchedulingRequest (already carrying fallback_strategy in actor_options).

# DeploymentSchedulingInfo
  bundle_label_selector: Optional[List[Dict[str, str]]] = None
+ fallback_strategy: Optional[List[Dict[str, Any]]] = None
  placement_group_strategy: Optional[str] = None

# DeploymentScheduler.on_deployment_deployed
  info.bundle_label_selector = replica_config.placement_group_bundle_label_selector
+ info.fallback_strategy = replica_config.ray_actor_options.get("fallback_strategy")
  info.max_replicas_per_node = replica_config.max_replicas_per_node

Related issue number

N/A

Checks

  • I've signed off every commit (git commit -s) in this PR.
  • I've run scripts/format.sh to lint the changes in this PR.
  • I've included any doc changes needed — N/A (internal field, no API surface).
  • I've made sure the tests are passing.
  • Testing Strategy
    • Unit tests
    • This PR is not tested :(

@johntaylor-cell johntaylor-cell requested a review from a team as a code owner June 25, 2026 18:02

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces tracking for the fallback_strategy in DeploymentSchedulingInfo and populates it from replica_config.ray_actor_options when a deployment is deployed. The reviewer suggests leveraging this newly tracked fallback_strategy to improve the downscaling logic in _get_replicas_to_stop by prioritizing stopping replicas on lower-priority fallback nodes.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

label_selector: Optional[Dict[str, str]] = None
placement_group_bundles: Optional[List[RequestedResources]] = None
bundle_label_selector: Optional[List[Dict[str, str]]] = None
fallback_strategy: Optional[List[Dict[str, Any]]] = None

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Now that fallback_strategy is tracked on DeploymentSchedulingInfo, we have an opportunity to improve the downscaling logic in _get_replicas_to_stop. Currently, _get_replicas_to_stop treats all replicas on fallback nodes equally (as non-matching). By leveraging the tracked fallback_strategy, we can prioritize stopping replicas on lower-priority fallback nodes (or default nodes) over higher-priority fallback nodes during downscaling.

@johntaylor-cell johntaylor-cell added the go add ONLY when ready to merge, run all tests label Jun 25, 2026
@ray-gardener ray-gardener Bot added the serve Ray Serve Related Issue label Jun 25, 2026
johntaylor-cell and others added 2 commits June 26, 2026 14:48
DeploymentSchedulingInfo records the actor label_selector and the
placement-group bundle_label_selector from on_deployment_deployed, but not
the actor fallback_strategy. Add it for symmetry so a scheduler that consults
the per-deployment DeploymentSchedulingInfo (rather than the per-replica
ReplicaSchedulingRequest) has access to the fallback strategy.

Additive only; no behavior change in the default deployment scheduler.

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: john.taylor <john.taylor@anyscale.com>
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: john.taylor <john.taylor@anyscale.com>
@johntaylor-cell johntaylor-cell force-pushed the serve-fallback-strategy-on-scheduling-info branch from 515ce27 to df24278 Compare June 26, 2026 14:48
@abrarsheikh abrarsheikh enabled auto-merge (squash) June 26, 2026 15:18
@abrarsheikh abrarsheikh merged commit 5afa89b into ray-project:master Jun 26, 2026
7 checks passed
limarkdcunha pushed a commit to limarkdcunha/ray that referenced this pull request Jun 30, 2026
…y-project#64346)

## Why are these changes needed?

`DeploymentScheduler.on_deployment_deployed` records a deployment's
scheduling constraints on `DeploymentSchedulingInfo` — including the
actor `label_selector` and the placement-group `bundle_label_selector` —
but it does not record the actor `fallback_strategy`. This adds a
`fallback_strategy` field to `DeploymentSchedulingInfo` and populates it
from `replica_config.ray_actor_options`, for parity with the other
recorded scheduling options.

This makes the actor fallback strategy available to schedulers that
reason over the per-deployment `DeploymentSchedulingInfo` rather than
the per-replica `ReplicaSchedulingRequest`. The change is purely
additive — one optional dataclass field plus its assignment in
`on_deployment_deployed`. There is **no behavior change** in
`DefaultDeploymentScheduler`, which builds placement candidates from the
`ReplicaSchedulingRequest` (already carrying `fallback_strategy` in
`actor_options`).

```python
# DeploymentSchedulingInfo
  bundle_label_selector: Optional[List[Dict[str, str]]] = None
+ fallback_strategy: Optional[List[Dict[str, Any]]] = None
  placement_group_strategy: Optional[str] = None

# DeploymentScheduler.on_deployment_deployed
  info.bundle_label_selector = replica_config.placement_group_bundle_label_selector
+ info.fallback_strategy = replica_config.ray_actor_options.get("fallback_strategy")
  info.max_replicas_per_node = replica_config.max_replicas_per_node
```

## Related issue number

N/A

## Checks

- [x] I've signed off every commit (`git commit -s`) in this PR.
- [ ] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed — N/A (internal field, no API
surface).
- [ ] I've made sure the tests are passing.
- [ ] Testing Strategy
  - [x] Unit tests
  - [ ] This PR is not tested :(

---------

Signed-off-by: john.taylor <john.taylor@anyscale.com>
Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

go add ONLY when ready to merge, run all tests serve Ray Serve Related Issue

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants