[serve] reduce remote calls for get handle APIs

[zcin]

[serve] reduce remote calls for get handle APIs

This PR makes two changes to the get handle APIs:

1. First, it optimizes serve.get_app_handle, which made two separate calls to the controller: one to get the ingress deployment name for the specified application, and one to check if the deployment exists. This is unnecessary and can be squashed into one call; in other words the second call doesn't need to be made.
2. serve.get_deployment_handle makes one call to the controller to check if the specified deployment exists. This remains the default behavior, but this PR adds a private flag _check_exists which can be set to False if the user wants to bypass that check for performance reasons.

Signed-off-by: Cindy Zhang cindyzyx9@gmail.com

[JoshKarpel]

Thanks for jumping on this so quickly @zcin , hugely appreciated!

While we're continuing to debug our performance issues, I think we spotted another place that could set check_exists=False, in the LongestPrefixRouter

ray/python/ray/serve/_private/proxy_router.py

Line 66 in 5902af8

handle = self._get_handle(endpoint.name, endpoint.app_name).options(

There's some layers of indirection here, but ultimately this update does come from the Serve Controller directly, if I understood the code correctly, so it probably doesn't need to check for deployment existence again.

[zcin]

Thanks for jumping on this so quickly @zcin , hugely appreciated!

While we're continuing to debug our performance issues, I think we spotted another place that could set check_exists=False, in the LongestPrefixRouter

ray/python/ray/serve/_private/proxy_router.py

Line 66 in 5902af8

handle = self._get_handle(endpoint.name, endpoint.app_name).options(

There's some layers of indirection here, but ultimately this update does come from the Serve Controller directly, if I understood the code correctly, so it probably doesn't need to check for deployment existence again.

@JoshKarpel Good catch, I believe that is an optimization we can make. I can address this in a separate PR!

[zcin]

@edoakes Is this ready to merge?

[JoshKarpel]

Thanks for jumping on this so quickly @zcin , hugely appreciated!
While we're continuing to debug our performance issues, I think we spotted another place that could set check_exists=False, in the LongestPrefixRouter

ray/python/ray/serve/_private/proxy_router.py

Line 66 in 5902af8

handle = self._get_handle(endpoint.name, endpoint.app_name).options(

There's some layers of indirection here, but ultimately this update does come from the Serve Controller directly, if I understood the code correctly, so it probably doesn't need to check for deployment existence again.

@JoshKarpel Good catch, I believe that is an optimization we can make. I can address this in a separate PR!

Awesome, sounds good!