A100: Client-side WRR slow start configuration #498

anuragagarwal561994 · 2025-05-31T04:49:04Z

No description provided.

ejona86 · 2025-06-02T14:20:30Z

anuragagarwal561994 · 2025-06-06T08:44:28Z

@dfawley did we get a chance to check this

atollena

This would be a nice addition (it's something we've had requests for internally at Datadog). Generally the approach sounds good to me. Ideally we would be able to support this mechanism for other load balancers supported by gRPC and Envoy (lr & rr). But given that lr & rr do not implement endpoint weights, it seems difficult to fit the feature in those balancers.

client-side-weighted-round-robin-slow-start.md

anuragagarwal561994 · 2025-06-06T18:55:08Z

@atollena I have addressed your comments regarding the proposal, let me know if I need to make any changes in it

anuragagarwal561994 · 2025-06-16T08:38:48Z

@ejona86 @atollena I have addressed the new comments, requesting review for any futher modifications needed

anuragagarwal561994 · 2025-06-19T15:02:46Z

@ejona86 can I start with the implementation of the feature?

anuragagarwal561994 · 2025-06-30T08:00:14Z

@ejona86 @atollena will it be possible to review the proposal this week so that I can begin with the implementation of the feature

atollena

It would be useful to see an implementation of this in one of the 3 languages. I wouldn't wait for the gRFC to be approved before starting, especially since the result can be used as a custom, private LB policy before it is merged to upstream-supported gRPC implementations.

client-side-weighted-round-robin-slow-start.md

atollena · 2025-07-01T11:24:13Z

client-side-weighted-round-robin-slow-start.md

+
+## Implementation
+
+This will be implemented in all languages C++, Java, and Go.


This is a good place to add the actual implementation of your choice when you have it ready (it can be a draft or closed PR that you can re-open once the gRFC has been approved).

I have added the implementation for Java, would love if you could review the changes :)

grpc/grpc-java#12200

client-side-weighted-round-robin-slow-start.md

anuragagarwal561994 · 2025-07-02T13:11:24Z

@atollena I have opened the respective MR at the envoy side as well, they will merge it once the proposal is approved at our end. For this proposal I just need to handle the case for the timing part and make the respective changes in the proposal once done I will assign for re-review.

Signed-off-by: anurag.ag <anuragagarwal561994@users.noreply.github.com>

anuragagarwal561994 · 2025-07-07T17:31:06Z

@atollena @ejona86 so I have created a rough implementation, not updated the test cases as of now.

I have also updated the proto on the envoy side, but to merge it we require to approve this proposal, then sync the proto may be in a separate MR.

How should we go about this?

atollena

👍

A100-client-side-weighted-round-robin-slow-start.md

Co-authored-by: Antoine Tollenaere <atollena@gmail.com>

dfawley

This mostly seems fine to me, I just have some nits. Also, since I don't see a slow start config in the envoy WRR policy proto, I'm curious if you've (or someone else has?) sent a proposal to add it there already.

A100-client-side-weighted-round-robin-slow-start.md

dfawley · 2025-07-17T21:28:27Z

A100-client-side-weighted-round-robin-slow-start.md

+
+## Metrics
+
+The following metric will be exposed to help monitor the slow start behavior:


How important is this metric to include? Should we consider not adding it until it's needed?

Ideally in most of the cases the slow_start_window will be applicable only for a short duration like 1-2 mins and if everything works well, it will not be called again.

I just thought that we can implement it in disabled mode, if someone wants to keeps a track of it.

We can decide this and I will make the respective changes in the proposal

anuragagarwal561994 · 2025-07-18T05:59:29Z

@dfawley envoyproxy/envoy#40090 this is the MR I have created at envoy's end to include the slow_start_config in proto.

However it requires this grpc proposal to be approved first before it is merged to the master.

Then we will need to sync across repo and then we are good to implement this feature. I have added sample MR with the java implementation as well, please do check it as well.

Co-authored-by: Doug Fawley <dfawley@google.com>

markdroth

Thanks for writing this up!

A100-client-side-weighted-round-robin-slow-start.md

markdroth · 2025-07-18T15:55:05Z

A100-client-side-weighted-round-robin-slow-start.md

+  //
+  // As time progresses, more and more traffic would be sent to endpoint, which is in slow start window.
+  // Once host exits slow start, time_factor and aggression no longer affect its weight.
+  google.protobuf.FloatValue aggression = 2;


Assuming 0 is not a valid value here, I think we can use float instead of google.protobuf.FloatValue.

@markdroth I am actually confused here on how to integrate this. Actually envoy already has this proto for SlowStartConfig which it uses for RR and LR load balancing.

Hence, there was no need of any changes in SlowStartConfig proto anywhere, the only change required was to inlcude it in the ClientSideWeightedRoundRobin proto at the envoy side.

So I think in an earlier suggestion on this proposal, I was asked to change it to a google type while envoy types were being used 60cc616

I just wanted to reference this proto for others to relate to the equations, but if we are going to use the envoy proto only then there we should make minimal changes for not confusing others. What would you suggest.

https://github.com/envoyproxy/envoy/blob/cdf3dd3b6e1f0a1274f097bd7f43350927b93f93/api/envoy/extensions/load_balancing_policies/common/v3/common.proto#L89-L113

markdroth · 2025-07-18T15:55:40Z

A100-client-side-weighted-round-robin-slow-start.md

+  // Configures the minimum percentage of the original weight that will be used for an endpoint
+  // in slow start. This helps to avoid a scenario in which endpoints receive no traffic during the
+  // slow start window. Must be between 0 and 100. If not specified, the default is 10%.
+  google.protobuf.UInt32Value min_weight_percent = 3;


If 0 is not a valid value here, I think we can use uint32 instead of google.protobuf.UInt32Value.

Same concern as stated above

A100-client-side-weighted-round-robin-slow-start.md

Signed-off-by: anurag.ag <anuragagarwal561994@users.noreply.github.com>

anuragagarwal561994 · 2025-07-20T10:43:35Z

@markdroth I have made the respective changes and added a few comments for where I was confused, please help me resolve the same.

anuragagarwal561994 marked this pull request as draft May 31, 2025 04:49

anuragagarwal561994 mentioned this pull request May 31, 2025

Support for slow start with WRR Load Balancer Policy grpc/grpc-java#12087

Open

anuragagarwal561994 marked this pull request as ready for review May 31, 2025 05:03

ejona86 requested review from ejona86 and markdroth June 2, 2025 14:20

atollena reviewed Jun 6, 2025

View reviewed changes

client-side-weighted-round-robin-slow-start.md Outdated Show resolved Hide resolved

client-side-weighted-round-robin-slow-start.md Outdated Show resolved Hide resolved

client-side-weighted-round-robin-slow-start.md Outdated Show resolved Hide resolved

anuragagarwal561994 requested a review from atollena June 22, 2025 01:09

atollena reviewed Jul 1, 2025

View reviewed changes

anuragagarwal561994 changed the title ~~Client-side WRR slow start configuration~~ A100: Client-side WRR slow start configuration Jul 1, 2025

anuragagarwal561994 force-pushed the wrrslowstart branch 2 times, most recently from 10be215 to 43b016a Compare July 1, 2025 12:39

This was referenced Jul 1, 2025

proto: Adds slow start config to wrr proto message envoyproxy/envoy#40090

Open

Slow Start Config support for Client Side WRR envoyproxy/envoy#40091

Open

anuragagarwal561994 added 9 commits July 2, 2025 20:01

Adds draft proposal for slow start config

c24c053

Signed-off-by: anurag.ag <anuragagarwal561994@users.noreply.github.com>

Deletes example

bd75f6d

Signed-off-by: anurag.ag <anuragagarwal561994@users.noreply.github.com>

Linking proposal

c294d61

Signed-off-by: anurag.ag <anuragagarwal561994@users.noreply.github.com>

Linking user

ce1be7d

Signed-off-by: anurag.ag <anuragagarwal561994@users.noreply.github.com>

Adds scopes and limitation section

e5e82df

Signed-off-by: anurag.ag <anuragagarwal561994@users.noreply.github.com>

Remove dependency from xds proto

60cc616

Signed-off-by: anurag.ag <anuragagarwal561994@users.noreply.github.com>

Updates proposal number to A100

54fbf1e

Signed-off-by: anurag.ag <anuragagarwal561994@users.noreply.github.com>

Updates references

f4af162

Signed-off-by: anurag.ag <anuragagarwal561994@users.noreply.github.com>

Updates proposal to include ready_since timestamp

e490283

Signed-off-by: anurag.ag <anuragagarwal561994@users.noreply.github.com>

anuragagarwal561994 force-pushed the wrrslowstart branch from 0a3ea16 to e490283 Compare July 2, 2025 14:32

anuragagarwal561994 requested a review from atollena July 2, 2025 14:37

anuragagarwal561994 mentioned this pull request Jul 2, 2025

Implementing SlowStartConfig for WRR Load Balancing Policy grpc/grpc-java#12200

Draft

Adds Java Implementation

3e6af14

Signed-off-by: anurag.ag <anuragagarwal561994@users.noreply.github.com>

atollena approved these changes Jul 17, 2025

View reviewed changes

A100-client-side-weighted-round-robin-slow-start.md Outdated Show resolved Hide resolved

Fix typo, making offers as offer

c80f1a5

Co-authored-by: Antoine Tollenaere <atollena@gmail.com>

dfawley reviewed Jul 17, 2025

View reviewed changes

anuragagarwal561994 and others added 2 commits July 18, 2025 11:36

Apply suggestions from code review

7ac1dc0

Co-authored-by: Doug Fawley <dfawley@google.com>

Fixes min_weight_percent nit

5201ee2

markdroth reviewed Jul 18, 2025

View reviewed changes

Updates proposal with slow start metrics and related references

8ff799f

Signed-off-by: anurag.ag <anuragagarwal561994@users.noreply.github.com>

anuragagarwal561994 requested review from markdroth, atollena and dfawley July 20, 2025 10:39

Updates proposal with experimental metric details and formatting fixes

e495afb

Signed-off-by: anurag.ag <anuragagarwal561994@users.noreply.github.com>


		## Implementation

		This will be implemented in all languages C++, Java, and Go.


		## Metrics

		The following metric will be exposed to help monitor the slow start behavior:

A100: Client-side WRR slow start configuration #498

Are you sure you want to change the base?

A100: Client-side WRR slow start configuration #498

Uh oh!

Conversation

anuragagarwal561994 commented May 31, 2025

Uh oh!

ejona86 commented Jun 2, 2025

Uh oh!

anuragagarwal561994 commented Jun 6, 2025

Uh oh!

atollena left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

anuragagarwal561994 commented Jun 6, 2025

Uh oh!

anuragagarwal561994 commented Jun 16, 2025

Uh oh!

anuragagarwal561994 commented Jun 19, 2025

Uh oh!

anuragagarwal561994 commented Jun 30, 2025

Uh oh!

atollena left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

anuragagarwal561994 commented Jul 2, 2025

Uh oh!

anuragagarwal561994 commented Jul 7, 2025

Uh oh!

atollena left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

dfawley left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

anuragagarwal561994 commented Jul 18, 2025

Uh oh!

markdroth left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!