[Feature] RFC-Serve: Support gRPC

[jiaodong]

Search before asking

• I had searched in the issues and found no similar feature requirement.

Description

We've got community asks about supporting gRPC in ray serve, as currently we're HTTP only. This issue opens up RFC for this request and open for community comments about their need and use case to help us prioritizing.

Use case

No response

Related issues

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

[satish-madugula]

Hi @jiaodong ,

It would be a good feature, if ray serve can support gRPC requests.

Comparison of gRPC and REST with TF-Serving

I had discussions on the same earlier , please refer the same here and here

[dynamicwebpaige]

+1 for supporting gRPC in Serve; this is something that Seldon Core supports today, and something that BentoML also has on its roadmap.

[bewestphal]

This'd be a big win for using cross-language clients in terms of our dev experience.

With GRPC, generating statically typed clients and interfaces helps us build resilient code across microservices.

[architkulkarni]

Experimental gRPC support has been added in #28175 and #29149 and is available in the nightly builds, please try it out and let us know your feedback!