-
Notifications
You must be signed in to change notification settings - Fork 10.5k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
1 changed file
with
96 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Original file line | Diff line number | Diff line change |
---|---|---|---|
@@ -0,0 +1,96 @@ | |||
Load Balancing in gRPC | |||
======================= | |||
|
|||
## Objective | |||
|
|||
To design a load balancing API between a gRPC client and a Load Balancer to | |||
instruct the client how to send load to multiple backend servers. | |||
|
|||
## Background | |||
|
|||
Prior to any gRPC specifics, we explore some usual way to approach load | |||
balancing. | |||
|
|||
## Proxy Model | |||
|
|||
Using a proxy provides a solid trustable client that can report load to the load | |||
balancing system. Proxies typically require more resources to operate since they | |||
have temporary copies of the RPC request and response. This model also increases | |||
latency to the RPCs. | |||
|
|||
The proxy model was deemed inefficient when considering request heavy services | |||
like storage. | |||
|
|||
## Balancing-aware Client | |||
|
|||
This thicker client places more of the load balancing logic in the client. For | |||
example, the client could contain many load balancing policies (Round Robin, | |||
Random, etc) used to select servers from a list. In this model, a load balancer | |||
would be responsible for providing a list of servers and let the client choose | |||
the preferred server from the list. | |||
|
|||
One of the drawbacks of this approach is writing and maintaining the load | |||
balancing policies in multiple languages and/or versions of the clients. These | |||
policies can be fairly complicated. Some of the algorithms also require client | |||
to server communication so the client would need to get thicker to support | |||
additional RPCs to get health or load information in addition to sending RPCs | |||
for user requests. | |||
|
|||
It would also significantly complicate the API: the new design hides the load | |||
balancing complexity of multiple layers and presents it as a simple list of | |||
servers to the client. | |||
|
|||
## External Load Balancing Service | |||
|
|||
The client load balancing code is kept simple and portable, implementing | |||
straightforward algorithms (ie, Pick First, Round Robin) for server selection. | |||
Complex load balancing algorithms are instead provided by the load balancer. The | |||
client relies on the load balancer to provide _load balancing configuration_ and | |||
_the list of servers_ to which the client should send requests. The balancer | |||
updates the server list as needed to balance the load as well as handle server | |||
unavailability or health issues. The load balancer will make any necessary | |||
complex decisions and inform the client. The load balancer may communicate with | |||
the backend servers to collect load and health information. | |||
|
|||
## Proposed Architecture | |||
|
|||
The gRPC load balancing approach follows the third approach, by having an | |||
external load balancer which provides simple clients with a list of servers. | |||
|
|||
## Client | |||
|
|||
When establishing a gRPC stream to the balancer, the client will send an initial | |||
request to the load balancer (via a regular gRPC message). The load balancer | |||
will respond with client config (including, for example, settings for flow | |||
control, RPC deadlines, etc.) or a redirect to another load balancer. If the | |||
balancer did not redirect the client, it will then send a list of servers to the | |||
client. The client will contain simple load balancing logic for choosing the | |||
next server when it needs to send a request. | |||
|
|||
## Load Balancer | |||
|
|||
The Load Balancer is responsible for providing the client with a list of servers | |||
and client RPC parameters. The balancer chooses when to update the list of | |||
servers and can decide whether to provide a complete list, a subset, or a | |||
specific list of “picked” servers in a particular order. The balancer can | |||
optionally provide an expiration interval after which the server list should no | |||
longer be trusted and should be updated by the balancer. | |||
|
|||
The load balancer is may open reporting streams to each server contained in the | |||
server list. These streams are primarily used for load reporting. For example, | |||
Weighted Round Robin requires that the servers report utilization to the load | |||
balancer in order to compute the next list of servers. | |||
|
|||
## Server | |||
|
|||
The gRPC Server is responsible for answering RPC requests and providing | |||
responses to the client. The server will also report load to the load balancer | |||
if a reporting stream was opened for this purpose. | |||
|
|||
### Security | |||
|
|||
The load balancer may be separate from the actual server backends and a | |||
compromise of the load balancer should only lead to a compromise of the | |||
loadbalancing functionality. In other words, a compromised load balancer should | |||
not be able to cause a client to trust a (potentially malicious) backend server | |||
any more than in a comparable situation without loadbalancing. |