[serve] Refresh GCS node-info cache off the control loop (async)

[johntaylor-cell]

At high replica counts with direct ingress, the controller's per-loop GCS node-info refresh (ClusterNodeInfoCache.update -> get_all_node_info / get_all_resource_usage) is a synchronous RPC on the event loop. When the query exceeds its deadline during cluster churn (~12K+ nodes/replicas) the blocking call freezes the control loop for the whole RPC, the cache goes stale, and direct-ingress / HAProxy backends empty out.

Add RAY_SERVE_ASYNC_NODE_INFO (default off): refactor update() into a pure _compute_snapshot() (no self-mutation, safe in a thread) + _apply_snapshot() (atomic assignment on the event-loop thread). When the flag is on, a background _node_info_refresh_loop runs _compute_snapshot via run_in_executor -- the GCS calls release the GIL (with nogil) so the control loop keeps running while a slow reply is in flight, and reads always see the last complete snapshot. One refresh in flight at a time; the synchronous per-step update() is gated off when async is on.

Behavior is unchanged with the flag off: update() still computes and applies the snapshot synchronously (same result as before). At high N it is safe to raise RAY_GCS_RPC_TIMEOUT_S once async is on, since a longer query no longer blocks the loop.

[gemini-code-assist]

Code Review

This pull request introduces an asynchronous mechanism to refresh the cluster node-info cache off the main control loop, preventing slow GCS queries from blocking the controller. The changes include adding a non-blocking refresh_async method to ClusterNodeInfoCache and running a background refresh loop in the controller when RAY_SERVE_ASYNC_NODE_INFO is enabled. The feedback recommends using run_background_task instead of asyncio.ensure_future to avoid garbage collection of the background task, utilizing the get_env_float_positive helper for environment variable parsing, and adding a type hint to the alive_id_set parameter.