Skip to content

original_dst: add batched host updates#38769

Closed
jronak wants to merge 1 commit intoenvoyproxy:mainfrom
jronak:jronak/original-dst
Closed

original_dst: add batched host updates#38769
jronak wants to merge 1 commit intoenvoyproxy:mainfrom
jronak:jronak/original-dst

Conversation

@jronak
Copy link
Contributor

@jronak jronak commented Mar 16, 2025

Commit Message: Add batched host updates in original_dst cluster

Additional Description: Adds batching support for host updates in original_dst clusters to address performance bottlenecks on the main thread. With high fan-out clients with large numbers of active connections (e.g. 60k) and high unique host-port connection rates (e.g. 3k/s), the current per-connection addHost callbacks create significant pressure on the main thread. Each callback triggers a host map copy and priority set update, which is CPU heavy and impacts critical operations like metrics collection + health checks on the main thread.

The new batch_update_interval configuration option allows consolidating these updates into periodic batches, significantly reducing main thread overhead. This has been production-tested for 6+ months with positive results.

Risk Level: Low
Testing: Unit tests
Docs Changes: proto docs
Release Notes: None
Platform Specific Features: None

Signed-off-by: Ronak Jain <ronakjainc@gmail.com>
@repokitteh-read-only
Copy link

CC @envoyproxy/api-shepherds: Your approval is needed for changes made to (api/envoy/|docs/root/api-docs/).
envoyproxy/api-shepherds assignee is @wbpcode
CC @envoyproxy/api-watchers: FYI only for changes made to (api/envoy/|docs/root/api-docs/).

🐱

Caused by: #38769 was opened by jronak.

see: more, trace.

Copy link
Member

@wbpcode wbpcode left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm api

@wbpcode wbpcode assigned kyessenov and unassigned wbpcode Mar 24, 2025
@wbpcode
Copy link
Member

wbpcode commented Mar 24, 2025

could you check the CI first, thanks?

@nezdolik
Copy link
Member

friendly ping @jronak, looks like CI needs fixing

@phlax
Copy link
Member

phlax commented Apr 15, 2025

@jronak needs main merge

@phlax
Copy link
Member

phlax commented Apr 15, 2025

/wait

@jronak
Copy link
Contributor Author

jronak commented Apr 16, 2025

@phlax ack, I'll rebase the PR soon. Apologies, I was unavailable for a week.

@github-actions
Copy link

This pull request has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in 7 days if no further activity occurs. Please feel free to give a status update now, ping for review, or re-open when it's ready. Thank you for your contributions!

@github-actions github-actions bot added the stale stalebot believes this issue/PR has not been touched recently label May 16, 2025
@github-actions
Copy link

This pull request has been automatically closed because it has not had activity in the last 37 days. Please feel free to give a status update now, ping for review, or re-open when it's ready. Thank you for your contributions!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

stale stalebot believes this issue/PR has not been touched recently waiting

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants