Added backend API and CLI for custom autoscaling

[vaishdho1]

Description

This PR introduces a backend API and CLI support for autoscaling observability. It reads and parses the autoscaling snapshot files (generated by #56225) to display detailed deployment level autoscaling status to the user

Changes:

• serve_head.py : Added a GET route that reads from the snapshot files, converts into a DeploymentAutoscalingDetail compatible schema and returns a JSON.
• schema.py : Added a helper constructor from_snapshot on DeploymentAutoscalingDetail to map raw snapshot data into a schema compliant object.
• scripts.py : Extended serve status with -v (and optional -d) to display deployment level autoscaling details entirely, per application or a specific deployment.
• Added a test that writes multiple entries across rotated snapshot files and assert that the API returns the latest autoscaling entry for each deployment.

Related issues

#56225 #55834

Checks

• I've signed off every commit (by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
• I've added any new APIs to the API Reference. For example, if I added a method in Tune, I've added it in doc/source/tune/api/ under the corresponding .rst file.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/

Testing Strategy

• Unit tests
• Release tests
• This PR is not tested :(

Sample outputs

Steps

• ray start --head
• ran a python code which initializes ray and populates serve output directory
• serve status -v

[gemini-code-assist]

Code Review

This pull request introduces a backend API and CLI for autoscaling observability, which is a valuable addition. The implementation is generally well-structured, including a new API endpoint, schema updates, and CLI extensions. The inclusion of a test case for the snapshot reading logic is also commendable. I have identified a few areas for improvement to enhance robustness and maintainability. Specifically, I've pointed out a potential ValueError in schema.py when handling timestamps, which could lead to a crash. I've also included suggestions for minor code cleanup in serve_head.py and a refactoring opportunity in scripts.py to improve readability. Overall, this is a solid contribution, and addressing these points will make the code even better.

[github-actions]

This pull request has been automatically marked as stale because it has not had
any activity for 14 days. It will be closed in another 14 days if no further activity occurs.
Thank you for your contributions.

You can always ask for help on our discussion forum or Ray's public slack channel.

If you'd like to keep this open, just leave any comment, and the stale label will be removed.

[vaishdho1]

With #56225 merged, I will edit this logic and update the PR

[harshit-anyscale]

@vaishdho1 please resolve the merge conflicts and microcheck failures.