[vtadmin-web] Add QPS and VReplicationQPS charts to Tablet view #8263
Conversation
Signed-off-by: Sara Bee <855595+doeg@users.noreply.github.com>
| export const formatTimeseriesMap = (rates: { [k: string]: number[] }, endAt?: number): TimeseriesMap => { | ||
| // Rates stats are (unfortunately) not returned with timestamps, so we infer them here. | ||
| // This behaviour matches that of the vtctld2 UI: | ||
| // https://github.com/vitessio/vitess/blob/main/go/vt/vttablet/tabletserver/status.go#L178 | ||
| const _endAt = typeof endAt === 'number' ? endAt : Date.now(); | ||
|
|
||
| // The `rates` map returned from the API, keyed by series name, will either: | ||
| // (a) be empty, or | ||
| // (b) contain a minimum of two series, one of them named "All". | ||
| // In the first case, inserting an empty "All" series renders more nicely | ||
| // on a Highcharts graph since it will include the axes, etc. So, we add it here. | ||
| const _rates = !!Object.keys(rates).length ? rates : { All: [] }; | ||
|
|
||
| const planTypes = Object.keys(_rates); | ||
|
|
||
| const data: TimeseriesMap = {}; | ||
|
|
||
| // This logic is borrowed from the vtctld2 UI | ||
| // https://github.com/vitessio/vitess/blob/09e74af/go/vt/vttablet/tabletmanager/vreplication/stats.go#L478-L505 | ||
| for (let i = 0; i < RATES_MAX_SPAN / RATES_INTERVAL; i++) { | ||
| const x = _endAt - ((i * 60) / RATES_INTERVAL) * 1000; | ||
|
|
||
| for (let j = 0; j < planTypes.length; j++) { | ||
| // Assume 0.0 QPS for older, non-existent data points. | ||
| let y = 0; | ||
|
|
||
| if (i < _rates[planTypes[j]].length) { | ||
| const idx = _rates[planTypes[j]].length - i - 1; | ||
| y = +_rates[planTypes[j]][idx].toFixed(2); | ||
| } | ||
|
|
||
| if (!Array.isArray(data[planTypes[j]])) { | ||
| data[planTypes[j]] = []; | ||
| } | ||
|
|
||
| data[planTypes[j]].unshift({ x, y }); | ||
| } | ||
| } | ||
|
|
||
| return data; |
There was a problem hiding this comment.
Apart from changing a couple of variable names and adding tests, I lifted this logic from the vtctld2 UI here: https://github.com/vitessio/vitess/blob/09e74af/go/vt/vttablet/tabletmanager/vreplication/stats.go#L478-L505
I came across two potential bugs/inconsistencies that I'll investigate further (outside this PR). In both cases, I decided to match the existing behaviour since at least that's consistent, for now.
-
The vtctld2 logic + comments state that the charts span a 15 minute time range, with data points at 5 second intervals. However, the x-axis on the existing charts (and therefore these new VTAdmin charts) spans ~ 35 minutes:
I think its the x-axis that is correct, and that the data is indeed doing a 15 minute span with a 5 second sampling interval, however I'll need to more closely investigate how rates work.
-
There seems to be a lurking bug in the aggregation of VReplicationQPS stats. It's tedious to replicate, but the VRep QPS data seems to flip-flop between individual stream rates every few (five?) seconds, rather than aggregating them. This behaviour has a similar feel to Stats: Rates object leaks goroutines #8171, although I'm not sure if it's related. Will follow up. :)
There was a problem hiding this comment.
doeg
added
Component: VTAdmin
Signed-off-by: Sara Bee 855595+doeg@users.noreply.github.com
Description
As the title says! This updates the Tablet view with QPS and VReplication QPS charts, which query the /debug/vars tablet endpoint.
I will almost certainly + indefinitely shuffle around the layout of the Tablet view but what else is new!
Here's what a cutover looks like. 🔪 ✨
Related Issue(s)
N/A
Checklist
Deployment Notes
N/A