Add panel for inflight object store requests to reads dashboard

[56quarters]

Signed-off-by: Nick Pillitteri nick.pillitteri@grafana.com

What this PR does

Add a new panel to the store-gateway section of the "Mimir / Reads" dashboard for the total number of inflight object store requests from the store-gateways.

Which issue(s) this PR fixes or relates to

N/A

Checklist

• Tests updated
• Documentation added
• CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

[56quarters]

Row screenshot

[replay]

I like the idea of seeing the number of inflight requests to the object store, but wouldn't it make more sense to put that on a row about the object store like for example Blocks object store (store-gateway accesses)? If it's on the row about the store-gateway it could easily be misinterpreted as the number of inflight requests to the store-gateway.

[56quarters]

I like the idea of seeing the number of inflight requests to the object store, but wouldn't it make more sense to put that on a row about the object store like for example Blocks object store (store-gateway accesses)? If it's on the row about the store-gateway it could easily be misinterpreted as the number of inflight requests to the store-gateway.

Yeah, I can see that. However, the code that emits this metric is part of a "gate" used to limit requests that is only used in the store-gateway (since the store-gateway is one of the few things doing significant operations on object storage).

If we added this to the object storage section of the dashboard, it would either be missing for the querier object store metrics or we'd have to add this gate to the queriers.

[replay]

If we added this to the object storage section of the dashboard, it would either be missing for the querier object store metrics or we'd have to add this gate to the queriers.

I'd just accept the fact that it is missing for the queriers and only add it to the store-gw object store metrics, as this reflects the reality of the metrics that we have. If someone is bothered by this, they could add the equivalent metrics to the queriers in the future. But this is just my opinion and I wouldn't be surprised if others disagree.

[56quarters]

If we added this to the object storage section of the dashboard, it would either be missing for the querier object store metrics or we'd have to add this gate to the queriers.

I'd just accept the fact that it is missing for the queriers and only add it to the store-gw object store metrics, as this reflects the reality of the metrics that we have. If someone is bothered by this, they could add the equivalent metrics to the queriers in the future. But this is just my opinion and I wouldn't be surprised if others disagree.

OK, that makes sense to me. I'll go ahead move it there.

[56quarters]

New screenshots

[replay]

Thanks for addressing my comments,
Looks great!