ateclient: spread RPCs over every ateapi replica - #1319
Merged
Eitan Yarmush (EItanya) merged 1 commit intoAug 30, 2026
Merged
Conversation
| // ateapi is a headless Service, so that is one address per replica, and gRPC's | ||
| // default of pick_first would send an entire client's traffic to whichever one | ||
| // it connected to first. internal/ateapiauth dials with the same policy. | ||
| const roundRobinServiceConfig = `{"loadBalancingConfig": [{"round_robin":{}}]}` |
Collaborator
Author
There was a problem hiding this comment.
This verbatim the config we use in internal/ateapiauth.
We could dedupe them but that seemed unnecessary for now. If this becomes more complex or we need N+1 client packages we probably should at that point.
Benjamin Elder (BenTheElder)
added
kind/bug
A client that asks for no load balancing policy gets gRPC's pick_first: it sends every RPC to whichever address it connected to first, for the life of that connection. So a client talks to one replica however many are running, and adding replicas moves no load. Measured with eight load generators against two replicas, one took 97% of the traffic. The policy is independent of where the addresses come from. internal/ateapiauth, which atelet and the controllers dial through, sets round_robin alongside the k8s EndpointSlice resolver for exactly this reason. ateclient resolves its target through gRPC's built-in resolver rather than that one, but either way the addresses are useless without a policy that spreads over them.
Benjamin Elder (BenTheElder)
force-pushed
the
ateclient-round-robin
branch
from
August 29, 2026 23:36
081e35b to
8f0228e
Compare
Eitan Yarmush (EItanya)
approved these changes
Aug 30, 2026
Eitan Yarmush (EItanya)
merged commit Aug 30, 2026
6cd878d
into
agent-substrate:main
9 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ateapi is a headless Service, so its DNS name resolves to one address per replica and the client picks. This one asked for no policy, which means gRPC's pick_first: it connects to whichever address answers first and sends everything there for the life of the connection.
So a client talks to one replica however many are running, and adding replicas moves no load. Measured with eight load generators against two replicas, one took 97% of the traffic.
internal/ateapiauth, which atelet and the controllers dial through, has asked for round_robin all along. This is the same policy for the clients that do not.
Pulled out from #1266, I ran into this while doing some custom benchmarking using ateclient, but this is already a bug for kubectl-ate, demos, ...
AI-assisted.