orca: add out-of-band reporting for client-side WRR by jukie · Pull Request #44403 · envoyproxy/envoy

jukie · 2026-04-12T17:32:54Z

Commit Message: orca: add out-of-band reporting for client-side WRR

Additional Description: Adds ORCA out-of-band load reporting support for client_side_weighted_round_robin. When enable_oob_load_report is true, Envoy opens a per-host HTTP/2 gRPC stream to xds.service.orca.v3.OpenRcaService/StreamCoreMetrics, decodes server-streamed OrcaLoadReports, and feeds them into the existing ORCA weight manager used by in-band reports. This also adds a dedicated host connection path for ORCA reporting, including LogicalHost address snapshotting, and cluster-scoped lb_orca_oob.* stats for reporting/session state.

Risk Level: Low
Testing: Added unit and integration coverage for:

ORCA OOB manager lifecycle, report decoding, transient failures, GOAWAY handling, and terminal UNIMPLEMENTED
host/logical-host ORCA reporting connection behavior
CSWRR OOB config defaults/overrides
basic OOB integration stats/report flow

Docs Changes: N/A
Release Notes: Added changelog entry for client_side_weighted_round_robin ORCA out-of-band reporting
Platform Specific Features: N/A
xRef #44318
xRef #44329

AI Disclosure: AI was used during implementation

repokitteh-read-only · 2026-04-12T17:33:00Z

As a reminder, PRs marked as draft will not be automatically assigned reviewers,
or be handled by maintainer-oncall triage.

Please mark your PR as ready when you want it to be reviewed!

🐱

Caused by: #44403 was opened by jukie.

see: more, trace.

jukie · 2026-04-12T18:23:50Z

/retest

adisuissa · 2026-04-14T11:42:06Z

Thanks for starting the work on this
I looked at #44318 but I'm not sure I understand the design. Specifically what are the requirements (e.g., how many OOB servers will be supported, how is an ORCA report shared between multiple servers on a different cadence), and what is the relationship between the different components.
Is there some doc that describes how the OOB feature of https://github.com/grpc/proposal/blob/master/A51-custom-backend-metrics.md will be incorporated into Envoy?

/assign @efimki

efimki · 2026-04-14T13:32:05Z

Tagging @markdroth to comment from gRPC perspective.

Reading https://github.com/grpc/proposal/blob/master/A51-custom-backend-metrics.md#out-of-band-reporting-1 it seems like in gRPC case server is pushing OOB load reports to connected clients.

jukie · 2026-04-14T14:57:47Z

From https://grpc.io/docs/guides/custom-backend-metrics/:

We're still matching the model where an upstream server pushes reports to connected clients (Envoy here) but a client needs to initiate the connection and communicate report_interval.

mathetake · 2026-04-20T15:27:19Z

kindly ping @efimki @adisuissa

jukie · 2026-04-20T16:15:11Z

I'll rework this and discuss the design a bit more in the issue

jukie · 2026-04-26T18:21:26Z

Reworked the implementation here based on feedback

jukie · 2026-04-26T20:21:35Z

/coverage

repokitteh-read-only · 2026-04-26T20:21:39Z

Coverage for this Pull Request will be rendered here:

https://storage.googleapis.com/envoy-cncf-pr/44403/coverage/index.html

For comparison, current coverage on main branch is here:

https://storage.googleapis.com/envoy-cncf-postsubmit/main/coverage/index.html

The coverage results are (re-)rendered each time the CI Envoy/Checks (coverage) job completes.

🐱

Caused by: a #44403 (comment) was created by @jukie.

see: more, trace.

jukie · 2026-04-26T23:21:17Z

/retest

jukie · 2026-04-26T23:32:13Z

Summary of the updated implementation:
OOB load reporting is wired into CSWRR as a separate OrcaOobManager alongside the existing OrcaWeightManager, modeled on GrpcHealthCheckerImpl. Both managers share OrcaLoadReportHandler, so OOB reports feed the same OrcaHostLbPolicyData atomics as the in-band path.

A new pure virtual Host::createOrcaReportingConnection (parallel to createHealthCheckConnection) is the seam for per-host codec clients; LogicalHost overrides for lock-snapshot semantics. The metadata + transport_socket_options parameters reserve room for a future per-host OobConfig override.
CSWRR wires enable_oob_load_report + oob_reporting_period. Manager is constructed only when enabled (default false → null member, zero overhead).
All OOB activity runs on the cluster's main-thread dispatcher; one stream per host. Sessions follow HC patterns for reset/GOAWAY/teardown; jittered exponential backoff with initial-attempt stagger; 3× reporting-period inactivity watchdog; Unimplemented is terminal.
Stats are cluster-scoped under lb_orca_oob - reports_received, report_errors, stream_failures, stream_terminated, active_sessions

jukie · 2026-04-27T00:03:35Z

/retest

jukie · 2026-04-28T21:51:10Z

Hey @paul-r-gall could you take another look at this?

Signed-off-by: jukie <10012479+jukie@users.noreply.github.com>

paul-r-gall · 2026-05-01T12:55:02Z

Requesting review from @yanavlasov or @botengyao for senior maintainer.

paul-r-gall · 2026-05-01T17:20:22Z

/gemini review

gemini-code-assist

Code Review

This pull request implements out-of-band (OOB) ORCA load reporting for the client_side_weighted_round_robin load balancing policy, introducing an OrcaOobManager to handle per-host gRPC streams. The implementation includes new connection creation logic in the Host interface and LogicalHost. Feedback points out that the authority() method's fallback to a string representation for certain address types, such as Unix Domain Sockets, could potentially lead to invalid characters in the HTTP :authority header.

Signed-off-by: jukie <10012479+jukie@users.noreply.github.com>

…ing for non-IP hosts Signed-off-by: jukie <10012479+jukie@users.noreply.github.com>

jukie · 2026-05-07T05:10:23Z

/gemini review

gemini-code-assist

Code Review

This pull request implements out-of-band (OOB) ORCA load reporting for the client-side weighted round-robin load balancing policy using server-streaming gRPC. It introduces the OrcaOobManager to manage per-host gRPC sessions and updates upstream host abstractions to support dedicated reporting connections. The review feedback highlights an issue with the :authority header formatting for IPv6 addresses, suggesting the use of asString() to ensure compliance with RFC 3986 by including brackets and the port number.

…ger.cc Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Signed-off-by: Isaac Wilson <isaac.wilson514@gmail.com>

paul-r-gall · 2026-05-13T11:52:41Z

/assign @botengyao

botengyao

sorry for the delay, and lgtm module one q.

botengyao · 2026-05-17T19:23:32Z

+
+void OrcaOobManager::OobSession::decodeData(Buffer::Instance& data, bool end_stream) {
+  std::vector<Grpc::Frame> frames;
+  if (!decoder_.decode(data, frames).ok()) {


Consider setting a max gRPC frame/message size? ORCA reports should be small, but this currently lets every upstream OOB stream buffer arbitrarily large frames, can be a follow-up.

Added a 64Ki limit

@botengyao updated. Please let me know if you have a different value in mind but I think 64KiB should be high enough. We can make this configurable once the per-host/cluster config surface is added for overrides.

sgtm, thanks for adding this.

Signed-off-by: jukie <10012479+jukie@users.noreply.github.com>

botengyao

lgtm, thanks!

Commit Message: orca: add out-of-band reporting for client-side WRR Additional Description: Adds ORCA out-of-band load reporting support for `client_side_weighted_round_robin`. When `enable_oob_load_report` is true, Envoy opens a per-host HTTP/2 gRPC stream to `xds.service.orca.v3.OpenRcaService/StreamCoreMetrics`, decodes server-streamed `OrcaLoadReport`s, and feeds them into the existing ORCA weight manager used by in-band reports. This also adds a dedicated host connection path for ORCA reporting, including LogicalHost address snapshotting, and cluster-scoped `lb_orca_oob.*` stats for reporting/session state. Risk Level: Low Testing: Added unit and integration coverage for: - ORCA OOB manager lifecycle, report decoding, transient failures, GOAWAY handling, and terminal `UNIMPLEMENTED` - host/logical-host ORCA reporting connection behavior - CSWRR OOB config defaults/overrides - basic OOB integration stats/report flow Docs Changes: N/A Release Notes: Added changelog entry for `client_side_weighted_round_robin` ORCA out-of-band reporting Platform Specific Features: N/A xRef envoyproxy#44318 xRef envoyproxy#44329 AI Disclosure: AI was used during implementation --------- Signed-off-by: jukie <10012479+jukie@users.noreply.github.com> Signed-off-by: Isaac Wilson <isaac.wilson514@gmail.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Signed-off-by: Alireza <alrzazz98@gmail.com>

jukie force-pushed the orca-oob-client-library branch from d940d58 to 5f1690d Compare April 12, 2026 18:03

jukie mentioned this pull request Apr 12, 2026

ORCA out-of-band-reporting #44318

Closed

jukie marked this pull request as ready for review April 13, 2026 22:39

phlax assigned adisuissa Apr 14, 2026

repokitteh-read-only Bot assigned efimki Apr 14, 2026

paul-r-gall reviewed Apr 14, 2026

View reviewed changes

Comment thread source/common/orca/orca_oob_client.cc Outdated

jukie requested a review from paul-r-gall April 15, 2026 01:08

mathetake added the waiting:any label Apr 20, 2026

jukie force-pushed the orca-oob-client-library branch from 00e398a to 939210a Compare April 26, 2026 18:15

jukie requested review from adisuissa, nezdolik, tonya11en and wbpcode as code owners April 26, 2026 18:15

repokitteh-read-only Bot removed the waiting:any label Apr 26, 2026

jukie changed the title ~~orca: add out of band reporting client library~~ orca: add out-of-band reporting for client-side WRR Apr 26, 2026

add orca oob manager

86a036d

Signed-off-by: jukie <10012479+jukie@users.noreply.github.com>

paul-r-gall previously approved these changes May 1, 2026

View reviewed changes

Comment thread source/common/upstream/upstream_impl.cc Outdated

gemini-code-assist Bot reviewed May 1, 2026

View reviewed changes

Comment thread source/extensions/load_balancing_policies/common/orca_oob_manager.cc

todo

f5e6635

Signed-off-by: jukie <10012479+jukie@users.noreply.github.com>

jukie dismissed paul-r-gall’s stale review via f5e6635 May 1, 2026 18:37

add orcaReportingAddress() to HostDescription and fix authority handl…

9a8d424

…ing for non-IP hosts Signed-off-by: jukie <10012479+jukie@users.noreply.github.com>

jukie requested a review from paul-r-gall May 7, 2026 05:13

gemini-code-assist Bot reviewed May 7, 2026

View reviewed changes

Comment thread source/extensions/load_balancing_policies/common/orca_oob_manager.cc

Update source/extensions/load_balancing_policies/common/orca_oob_mana…

61304ae

…ger.cc Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Signed-off-by: Isaac Wilson <isaac.wilson514@gmail.com>

paul-r-gall previously approved these changes May 13, 2026

View reviewed changes

repokitteh-read-only Bot assigned botengyao May 13, 2026

kyessenov unassigned efimki and adisuissa May 13, 2026

botengyao reviewed May 17, 2026

View reviewed changes

Add max frame size

c5eda24

Signed-off-by: jukie <10012479+jukie@users.noreply.github.com>

jukie dismissed paul-r-gall’s stale review via c5eda24 May 18, 2026 05:12

jukie added 2 commits May 17, 2026 23:48

Merge remote-tracking branch 'origin/main' into orca-oob-client-library

055896e

Signed-off-by: jukie <10012479+jukie@users.noreply.github.com>

Use non-deprecated waitForCounter/waitForGauge

3cfae49

Signed-off-by: jukie <10012479+jukie@users.noreply.github.com>

jukie requested review from botengyao and paul-r-gall May 18, 2026 13:42

botengyao approved these changes May 19, 2026

View reviewed changes

botengyao merged commit ae96a25 into envoyproxy:main May 19, 2026
29 checks passed

jukie deleted the orca-oob-client-library branch May 19, 2026 14:11

This was referenced Jun 12, 2026

cswrr: add oob_reporting_config for ORCA OOB overrides #45235

Merged

load_balancing: add load-aware locality LB policy proto, docs, and config stub #44341

Merged

jukie mentioned this pull request Jul 1, 2026

Support ORCA Out Of Band Reporting on BackendUtilization LoadBalancer envoyproxy/gateway#9401

Open

Uh oh!

Conversation

jukie commented Apr 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

repokitteh-read-only Bot commented Apr 12, 2026

Uh oh!

jukie commented Apr 12, 2026

Uh oh!

adisuissa commented Apr 14, 2026

Uh oh!

efimki commented Apr 14, 2026

Uh oh!

jukie commented Apr 14, 2026

Uh oh!

Uh oh!

mathetake commented Apr 20, 2026

Uh oh!

jukie commented Apr 20, 2026

Uh oh!

jukie commented Apr 26, 2026

Uh oh!

jukie commented Apr 26, 2026

Uh oh!

repokitteh-read-only Bot commented Apr 26, 2026

Uh oh!

jukie commented Apr 26, 2026

Uh oh!

jukie commented Apr 26, 2026

Uh oh!

jukie commented Apr 27, 2026

Uh oh!

jukie commented Apr 28, 2026

Uh oh!

Uh oh!

paul-r-gall commented May 1, 2026

Uh oh!

paul-r-gall commented May 1, 2026

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

jukie commented May 7, 2026

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

paul-r-gall commented May 13, 2026

Uh oh!

botengyao left a comment

Choose a reason for hiding this comment

Uh oh!

botengyao May 17, 2026

Choose a reason for hiding this comment

Uh oh!

jukie May 18, 2026

Choose a reason for hiding this comment

Uh oh!

jukie May 18, 2026

Choose a reason for hiding this comment

Uh oh!

botengyao May 19, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

botengyao left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

jukie commented Apr 12, 2026 •

edited

Loading