[serve] Translate gRPC UNAVAILABLE into ActorUnavailableError in done-callback

[isotauon]

Why are these changes needed?

Closes #63261.

When a DeploymentHandle uses _by_reference=False (gRPC transport), gRPCReplicaResult.add_done_callback previously registered the user's callback directly on the underlying grpc.aio.Call:

def add_done_callback(self, callback: Callable):
    self._call.add_done_callback(callback)

grpc.aio.Call invokes its done-callbacks with the call object itself, not the deserialized result. The router's AsyncioRouter._process_finished_request dispatches on the Ray error type of its result argument:

actor_died_error = self._get_actor_died_error(result)
if actor_died_error is not None: ...
elif isinstance(result, ActorUnavailableError):
    self.request_router.on_replica_actor_unavailable(replica_id)

So on the gRPC path neither error branch fired, the queue-length cache was never invalidated for a failed replica, and power-of-two-choices kept routing to that replica until either the next request's rejection path or a controller long-poll pushed a refresh.

Fix

Wrap the user callback in a translating shim. When the underlying call is done and not cancelled, inspect call.exception(): an AioRpcError with StatusCode.UNAVAILABLE is translated into an ActorUnavailableError — matching what ActorReplicaResult.add_done_callback delivers for the actor transport, and matching the existing pattern already used elsewhere in replica_result.py (e.g. replica_result.py:374-379, replica_result.py:506-510).

if call.done() and not call.cancelled():
    exc = call.exception()
    if isinstance(exc, grpc.aio.AioRpcError) and exc.code() == grpc.StatusCode.UNAVAILABLE:
        translated = ActorUnavailableError(
            "Actor is unavailable.",
            self._actor_id.binary(),
        )

Cancellations and other status codes pass the call through unchanged so existing callers are unaffected. Failures during translation are logged and the call is passed through (no callback chain breakage).

Scope intentionally limited to UNAVAILABLE

Only UNAVAILABLE is translated, matching the existing precedent at lines 374-379 and 506-510. INTERNAL / UNKNOWN could in theory indicate replica issues too, but they can also reflect application-level errors that aren't the router's concern. Happy to widen the translation if maintainers prefer.

Related issue number

Closes #63261

Checks

• I've signed off every commit (by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR. (verified manually; new code stays within the file's existing line-width budget)
• I've included any doc changes needed for https://docs.ray.io/en/master/. (no doc changes needed — bugfix in internal behavior)
• I've added any new APIs to the API Reference. (N/A — no new public APIs)
• I've made sure the tests are passing. (new unit tests added in python/ray/serve/tests/unit/test_grpc_replica_result.py; not run locally because the local environment has no Ray install. Verified python -m py_compile on the changed files; relies on CI to confirm the existing suite still passes.)
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[gemini-code-assist]

Code Review

This pull request modifies gRPCReplicaResult.add_done_callback to translate gRPC UNAVAILABLE status codes into ActorUnavailableError. This change ensures a consistent error contract for routers, allowing them to handle transport-level failures and actor-path failures uniformly for cache invalidation. Corresponding unit tests were added to verify the translation logic for various gRPC call outcomes. I have no feedback to provide.