Skip to content

Surface per-server stats on BrokerResponse for SSE queries#18602

Merged
xiangfu0 merged 1 commit into
apache:masterfrom
st-omarkhalid:add-server-stats-to-broker-response
May 28, 2026
Merged

Surface per-server stats on BrokerResponse for SSE queries#18602
xiangfu0 merged 1 commit into
apache:masterfrom
st-omarkhalid:add-server-stats-to-broker-response

Conversation

@st-omarkhalid
Copy link
Copy Markdown
Contributor

@st-omarkhalid st-omarkhalid commented May 28, 2026

Expose serverStats in SSE broker response.

@st-omarkhalid st-omarkhalid force-pushed the add-server-stats-to-broker-response branch from 046e4e4 to 735c830 Compare May 28, 2026 01:07
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented May 28, 2026

Codecov Report

❌ Patch coverage is 80.00000% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 64.37%. Comparing base (9474069) to head (a87229e).
⚠️ Report is 6 commits behind head on master.

Files with missing lines Patch % Lines
...sthandler/BaseSingleStageBrokerRequestHandler.java 50.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #18602      +/-   ##
============================================
+ Coverage     64.27%   64.37%   +0.09%     
  Complexity     1137     1137              
============================================
  Files          3335     3336       +1     
  Lines        206006   206101      +95     
  Branches      32133    32145      +12     
============================================
+ Hits         132420   132685     +265     
+ Misses        62940    62758     -182     
- Partials      10646    10658      +12     
Flag Coverage Δ
custom-integration1 100.00% <ø> (?)
integration 100.00% <ø> (ø)
integration1 100.00% <ø> (ø)
integration2 0.00% <ø> (ø)
java-21 64.37% <80.00%> (+0.09%) ⬆️
temurin 64.37% <80.00%> (+0.09%) ⬆️
unittests 64.37% <80.00%> (+0.09%) ⬆️
unittests1 56.77% <100.00%> (-0.02%) ⬇️
unittests2 36.93% <80.00%> (+0.11%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Jackie-Jiang Jackie-Jiang added query Related to query processing observability Related to observability (logging, tracing, metrics) labels May 28, 2026
Comment thread pinot-common/src/main/java/org/apache/pinot/common/response/BrokerResponse.java Outdated
@st-omarkhalid st-omarkhalid force-pushed the add-server-stats-to-broker-response branch from 735c830 to 2620e97 Compare May 28, 2026 13:03
@st-omarkhalid
Copy link
Copy Markdown
Contributor Author

st-omarkhalid commented May 28, 2026

Good call — moved off the interface. Latest push:

  • Removed the default String getServerStats() from BrokerResponse.
  • Removed @Override from BrokerResponseNative.getServerStats().

serverStats now lives only on BrokerResponseNative (the SSE response class), mirroring how BrokerResponseNative reference and call its getter directly — same asymmetry as setMaterializedViewQueried.

Tests still 8/8 pass; checkstyle clean.

@st-omarkhalid st-omarkhalid force-pushed the add-server-stats-to-broker-response branch from 2620e97 to 1754a71 Compare May 28, 2026 13:21
Per-server scatter stats (submit delay, response delay, response size,
deserialization timing) are computed by the Netty single-connection
transport during scatter and stashed on a local ServerStats holder
inside BaseSingleStageBrokerRequestHandler.handleRequest. Today the
formatted string only reaches the SLF4J QueryLogger that consumes the
holder before handleRequest returns; subscribers of the
onQueryCompletion(RequestContext, BrokerResponse) hook and JSON-API
clients have no way to read it because neither RequestContext nor
BrokerResponse exposes it.

This commit exposes the string on BrokerResponse, mirroring the existing
materializedViewQueried pattern: default getter on the BrokerResponse
interface (returns null so external implementations need no override),
concrete field + setter + Jackson-annotated getter on
BrokerResponseNative, and @JsonInclude(NON_NULL) so existing clients
that do not expect the field see no behavioral change.
BaseSingleStageBrokerRequestHandler#handleRequest writes the value onto
the response after scatter completes, both on the regular path and the
materialized-view split path. Only the Netty SSE transport produces a
non-null value today; gRPC SSE and MSE leave it null because OSS does
not currently compute a per-server stats string on those paths.

Tests in BrokerResponseNativeTest pin (1) JSON round-trip, (2)
suppression of the field when null, and (3) the null default.
@st-omarkhalid st-omarkhalid force-pushed the add-server-stats-to-broker-response branch from 1754a71 to a87229e Compare May 28, 2026 13:25
@xiangfu0 xiangfu0 merged commit 405ad46 into apache:master May 28, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

observability Related to observability (logging, tracing, metrics) query Related to query processing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants