numServersQueried, responded is not set in BrokerResponse for MSQE

[vrajat]

Reproduce:

• Start MultiStageQueryEngineQuickStart.
• Run the following commands:

# Run query using SSE
 curl --request POST http://localhost:9000/sql --data '{"sql":"SELECT playerName, teamID FROM baseballStats_OFFLINE LIMIT 10", "queryOptions":"useMultistageEngine=false"}' | jq > /tmp/sse

# Run query using MSE
curl --request POST http://localhost:9000/sql --data '{"sql":"SELECT playerName, teamID FROM baseballStats_OFFLINE LIMIT 10", "queryOptions":"useMultistageEngine=true"}' | jq > /tmp/mse

The relevant portions of diff are:

 diff /tmp/sse /tmp/mse

67,68c131,132
<   "numServersQueried": 1,
<   "numServersResponded": 1,
---
>   "numServersQueried": 0,
>   "numServersResponded": 0,

Note that lines starting with > are from /tmp/mse

Other fields are set to 0 as well but it is not clear to me if the time taken was > 0 ms.

81,82c145,146
<   "brokerReduceTimeMs": 0,
<   "offlineThreadCpuTimeNs": 129000,
---
>   "brokerReduceTimeMs": 2,
>   "offlineThreadCpuTimeNs": 0,
84c148
<   "offlineSystemActivitiesCpuTimeNs": 44042,
---
>   "offlineSystemActivitiesCpuTimeNs": 0,
86c150
<   "offlineResponseSerializationCpuTimeNs": 9000,
---
>   "offlineResponseSerializationCpuTimeNs": 0,
88c152
<   "offlineTotalCpuTimeNs": 182042,
---
>   "offlineTotalCpuTimeNs": 0,

[yashmayya]

Looks like this was left as a TODO -

pinot/pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/MultiStageBrokerRequestHandler.java

Line 256 in b4066fb

// TODO: Add servers queried/responded stats

[yashmayya]

It should be fairly straightforward to get the servers queried from the query plan in the broker. I'm not sure if there's a straightforward way to get the servers responded stats though.

[gortiz]

It should be fairly straightforward to get the servers queried from the query plan in the broker. I'm not sure if there's a straightforward way to get the servers responded stats though.

I think in MSE there is now way a query can finish successfully if a queried server doesn't respond. So in successful queries we can always return the same value used in server queries. What I don't think it is worth it is to try to calculate how many servers didn't answer. That makes sense in SSE where users can choose availability over consistency.