Support xDS in Tonic and gRPC

[gu0keno0]

Feature Request

Support xDS in Tonic and gRPC

Motivation

In recent years, the official gRPC implementations have been adding out-of-box service discovery and load-balancing capabilities via the xDS protocol. This enables gRPC users to build a service mesh without having to pay the overhead of proxies: by talking to a xDS control plane (e.g. Istio, Google Cloud Traffic Director), gRPC client can automatically discover gRPC servers, route traffic to them and dynamically configure its load-balancing algorithms out of the box.

My team have been using gRPC and xDS massively in our production environment. For Rust, we have recently added xDS support to our internal Tonic-based gRPC framework, and it achieved significant performance improvements over the Proxy-based service mesh implementation. We’d like to contribute our xDS implementation to Tonic as well as the ongoing Rust gRPC effort, so that other Rust users can also benefit from the lightweight proxyless service mesh.

Proposal

The core components of our current solution are:

1. xDS client for subscribing resources to the xDS control plane, it is used by the Tower layers in #3
2. A “Cluster Channel” that leverages the tower::discover for dynamically configuring the endpoints under a xDS EDS cluster and tower::balance for load balancing among them
3. A few Tower layers on top of Cluster Channel for implementing the follows:
   a. Service Discovery & Routing (LDS → RDS → CDS)
   b. Handling retries
   c. Cluster Channel selection and dispatching

On top of this basic work, we are also actively adding other xDS features such as RDS routing rules, outlier detection, sticky routing, etc. For the xDS client, we plan to support delta xDS protocol and LEDS to handle our large fleet.

Per our previous discussions, we could potentially first contribute the xDS client and provide a generic mechanism for subscribing typed resources to xDS control plane. The APIs can be made transport agnostic, potentially similar to the Go gRPC XDSClient interface . The xDS client can be shared by both Tonic and Rust gRPC code.

Once the xDS client is available, we could potentially open-source the Tower Layers as well. Meanwhile, we are open to collaborating on building the xDS-based Resolver and Picker implementations for gRPC as well. There are some big differences between the Tower-based architecture and Rust gRPC, which does not use Tower at all. We could potentially shape the core routing and load-balancing logics to be close to or even directly implementing Resolver and Picker traits so that we can leverage it in both Tower-based implementation (short-term) and gRPC (long-term).

We are looking forward to further discussions, and can POC to demonstrate some of the mentioned ideas as well.

cc: @dfawley @LucioFranco

[LucioFranco]

Once the xDS client is available, we could potentially open-source the Tower Layers as well.

How much complexity would it be to ship this in the first iteration?

I think what should maybe happen would be to ship the first iteration using the current tonic/tower approach and then start to extract the core mechanics into something that can later be used with the future iterations of the traits for grpc-rust. This provides the most flexibility and impact right away. From my understanding this approach should also be the easiest since the project is already in a tonic/tower approach?

[gu0keno0]

How much complexity would it be to ship this in the first iteration?

It should be straightforward we will probably need to add interfaces for exporting metrics, it's currently bound to OTEL, but the rest is all based on OSS Tonic.

I think what should maybe happen would be to ship the first iteration using the current tonic/tower approach and then start to extract the core mechanics into something that can later be used with the future iterations of the traits for grpc-rust.

Agreed. Our current implementation is architectured in a way close to Google's Resolver and Picker interfaces, so I'd like to see if in the OSS contribution we can structure the code that can be reused by rust-grpc in the future.