Add query bytes fetched along with other series stats in Query Response

[christopherzli]

Add query bytes fetched along with other series stats in Query Response
Want to seek approval before I added changelog

• I added CHANGELOG entry for this change.
• Change is not relevant to the end user.

Changes

Verification

curl http://localhost:10902/api/v1/query?query='COUNT(up)'
{"status":"success","data":{"resultType":"vector","result":[{"metric":{},"value":[1716515107.878,"173"]}],"analysis":{},"seriesStatsCounter":{"Series":174,"Chunks":174,"Samples":2248,"Bytes":101533}}}

[christopherzli]

it is to address this issue: #7365

[yeya24]

It is a change on API response. I wonder if we want to put this under a FF in the API request.

[christopherzli]

It is a change on API response. I wonder if we want to put this under a FF in the API request.

sure that sounds like an idea. Another route I want to chase down is if we want to include in other metadata format like http headers, do we have any guidelines/examples on where we should put?

[yeya24]

Sorry I am a little bit unsure what you want to achieve here with http headers. Can you elaborate it more?

Do you plan to log the info or ?

[christopherzli]

Our use case is to export this stat information along with query response, and it could be part of response or pass with http headers.
We want to pass along that information to query frontend and finally return as part of http response or header as well so that we can do our query cost attribution to see how much data is fetched for each query.
We want to seek suggestion on what is the best option to export this piece of information for query cost attribution.
We previously use M3DB as our metrics engine and there was query bytes fetched information in their response header.
The reason I put that in part of Thanos query data response struct instead of http header is that I do not see any existing use of http header as part of query response.

[GiedriusS]

I'm against this, it's a feature that could be so much more useful. For example, if you have a query like sum(a) + sum(b) then how do you know which selector downloaded more data and what should be the focus of optimizations? We could do much better. In fact, there's a mechanism/place where this kind of info should be exposed.

Please take a look at the Analyze functionality and focus there. A good plan of action would be first to add some mechanism through context.Context that would allow capturing more granular data about the fan-out. This could be used in the PromQL engine to "surface" this data through the Analyze() call, and then this could be shown in the UI next to each selector.

[christopherzli]

Hi @GiedriusS in our case our usecase is much simpler: we simply want to know how expensive each query is and attribute our total query cost to each user based on # of bytes fetched/# of timeseries.
I could follow @yeya24 's suggestion on making it a command line flag on querier.
The method you mentioned is an overkill for this usecase and could be put as a TODO, what do you think?