Skip to content

Comments

Implement node based routing table#6159

Merged
nadav-govari merged 9 commits intonadav/feature-node-based-routingfrom
nadav/pr2
Feb 19, 2026
Merged

Implement node based routing table#6159
nadav-govari merged 9 commits intonadav/feature-node-based-routingfrom
nadav/pr2

Conversation

@nadav-govari
Copy link
Collaborator

@nadav-govari nadav-govari commented Feb 13, 2026

This table will not be in production use as of this PR. It's added alongside the existing one and will be developed incrementally. Once everything is ready, routing will be switched over to this one.

Description

This PR creates a node based routing table alongside the existing one. The new table isn't used yet. This creates the APIs for loading it with data, and for picking nodes, including the power of two choices implementation.

It's added to the router's state, which subscribes to the new IngesterCapacityUpdate broadcast and uses it to update the table entries.

There are additional changes required before this can be landed in production.

  • Remove shard references from the Persist protos.
  • Modify error handling: rate limiting, closed shards go away.
  • Remove closed shard reporting in GetOrCreateOpenShards.
  • Report capacity scores in GetOrCreateOpenShards responses.
  • Use the new table in routing decisions.

How was this PR tested?

New unit tests were added.

These changes are not yet ready for live testing.

@nadav-govari nadav-govari force-pushed the nadav/pr2 branch 2 times, most recently from 6441132 to 745e73e Compare February 13, 2026 20:32
Base automatically changed from nadav/node-affinity-broadcast to nadav/feature-node-based-routing February 17, 2026 16:45
Copy link
Member

@guilload guilload left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please address my comments, but good to merge in your other branch.

@nadav-govari nadav-govari merged commit 6b22b1d into nadav/feature-node-based-routing Feb 19, 2026
4 checks passed
@nadav-govari nadav-govari deleted the nadav/pr2 branch February 19, 2026 18:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants