mgr/dashboard: monitoring:Implement BlueStore onode hit/miss counters into the dashboard

[aaSharma14]

Provide the details pulled from Bluestore stats in order to display the onode hit/miss counters

Depends on #44334
Fixes: https://tracker.ceph.com/issues/53577
Signed-off-by: Aashish Sharma aasharma@redhat.com

Show available Jenkins commands

• jenkins retest this please
• jenkins test classic perf
• jenkins test crimson perf
• jenkins test signed
• jenkins test make check
• jenkins test make check arm64
• jenkins test submodules
• jenkins test dashboard
• jenkins test dashboard cephadm
• jenkins test api
• jenkins test docs
• jenkins render docs
• jenkins test ceph-volume all
• jenkins test ceph-volume tox

[avanthakkar]

Some unit-test would be nice to have @aaSharma14

[aaSharma14]

Some unit-test would be nice to have @aaSharma14

Done. Thanks @avanthakkar

[pereman2]

jenkins test make check

[pereman2]

Looks good

[aaSharma14]

jenkins test make check

[epuertat]

@neha-ojha @ljflores @markhpc folks, what do you think about this one?

[alfonsomthd]

jenkins test dashboard

[aaSharma14]

jenkins test dashboard

[ljflores]

Looks great!

I want to mention that there is another PR open right now (#44334) that shortens the names of the BlueStore onode hit/miss counters when printed. I don't think that would affect this PR, but linking it just in case.

[aaSharma14]

Looks great!

I want to mention that there is another PR open right now (#44334) that shortens the names of the BlueStore onode hit/miss counters when printed. I don't think that would affect this PR, but linking it just in case.

Thanks @ljflores , Yes, in this case we need to change the metrics name here in this PR as well. We can merge this one after #44334 gets merged.

[ljflores]

@aaSharma14 the branch needs to be updated

[aaSharma14]

jenkins test dashboard cephadm

[ljflores]

@epuertat sounds good

[neha-ojha]

looks great!

[epuertat]

Now I check the time-series chart again, being both series monotonic counters, I realize that if the hit/miss ratio is uneven (let's say 90% hit ratio, or 90% miss ratio), over time the smallest metric will be almost zero, and a temporal chart won't be very informative.

A possibility would be to display the hit-miss ratio over time. There's this Grafana Stat Panel widget that we don't use, and allows us printing a percentage (e.g.: hit ratio), a trend in the background, and also a threshold-driven color.

And coming back to the pie chart, we could replace it with the built-in gauge panel (which is great for displaying a binary dataset). One issue with pie-charts (vs. doughnut charts or gauges), is that for conveying information they rely on areas, and areas are squared values, so they actually display information^2. Gauges also allow defining thresholds, so we can visually tell users whether a 20% hit rate is actually a good (green) or bad (red) value.

So, summarizing my point:

• Can we have 1 new chart instead of 2?
• Can that be a gauge (if we don't mind temporal trends) or a Stat Panel otherwise?

[aaSharma14]

Now I check the time-series chart again, being both series monotonic counters, I realize that if the hit/miss ratio is uneven (let's say 90% hit ratio, or 90% miss ratio), over time the smallest metric will be almost zero, and a temporal chart won't be very informative.

A possibility would be to display the hit-miss ratio over time. There's this Grafana Stat Panel widget that we don't use, and allows us printing a percentage (e.g.: hit ratio), a trend in the background, and also a threshold-driven color.

And coming back to the pie chart, we could replace it with the built-in gauge panel (which is great for displaying a binary dataset). One issue with pie-charts (vs. doughnut charts or gauges), is that for conveying information they rely on areas, and areas are squared values, so they actually display information^2. Gauges also allow defining thresholds, so we can visually tell users whether a 20% hit rate is actually a good (green) or bad (red) value.

So, summarizing my point:

• Can we have 1 new chart instead of 2?
• Can that be a gauge (if we don't mind temporal trends) or a Stat Panel otherwise?

Thank you for the suggestion @epuertat . I am currently implementing a gauge panel for this one. In any case what is the ideal onode Hits ratio ? so we can display a warning or error scenario if the ratio goes below that value. Eg. I have kept it 50% as of now, so if the ratio goes below 50%, it would show that in red color and otherwise green. @ljflores @neha-ojha Do you guys have some insight about this?

[ljflores]

In any case what is the ideal onode Hits ratio ?

@aaSharma14 @epuertat Just talked with @aclamk about this. He said he uses the threshold of 75%, where above that is fine, and below that is something to investigate.

This is looking great!

Edit: I switched the "above" and "below" scenarios. Apologies, the comment is correct now.

[aaSharma14]

In any case what is the ideal onode Hits ratio ?

@aaSharma14 @epuertat Just talked with @aclamk about this. He said he uses the threshold of 75%, where above that is fine, and below that is something to investigate.

This is looking great!

Edit: I switched the "above" and "below" scenarios. Apologies, the comment is correct now.

Thankyou @ljflores for the information, I have updated the PR accordingly.

[aaSharma14]

jenkins test dashboard

[aaSharma14]

jenkins test make check

[aaSharma14]

jenkins test api

[aaSharma14]

jenkins test make check