kv: add observability for number of accurate- vs. approximate-stats splits #119516
Assignees
Labels
A-kv
Anything in KV that doesn't belong in a more specific category.
C-enhancement
Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)
T-kv
KV Team
Projects
Comments
miraradeva
added
C-enhancement
miraradeva
added a commit
to miraradeva/cockroach
that referenced
this issue
Mar 6, 2024
This patch adds logging and two new metrics to track estimated MVCC stats computed during splits. `kv.split.estimated_stats`: the number of splits that computed estimated MVCC stats, as opposed to 100% accurate ones. `kv.split.total_bytes_estimates`: the number of total bytes of estimates introduced by splits. These are calculated as the difference between the pre-computed stats before the split and the stored stats during the split (while holding latches). Fixes: cockroachdb#119516 Release note: None
miraradeva
added a commit
to miraradeva/cockroach
that referenced
this issue
Mar 11, 2024
This patch adds logging and two new metrics to track estimated MVCC stats computed during splits. `kv.split.estimated_stats`: the number of splits that computed estimated MVCC stats, as opposed to 100% accurate ones. `kv.split.total_bytes_estimates`: the number of total bytes of estimates introduced by splits. These are calculated as the difference between the pre-computed stats before the split and the stored stats during the split (while holding latches). Fixes: cockroachdb#119516 Release note (ops-change): Two new metrics (kv.split.estimated_stats and kv.split.total_bytes_estimates) added to track the number of splits that produce MVCC stats estimates, and the total bytes of estimates produced.
miraradeva
added a commit
to miraradeva/cockroach
that referenced
this issue
Mar 14, 2024
This patch adds logging and two new metrics to track estimated MVCC stats computed during splits. `kv.split.estimated_stats`: the number of splits that computed estimated MVCC stats, as opposed to 100% accurate ones. `kv.split.total_bytes_estimates`: the number of total bytes of estimates introduced by splits. These are calculated as the difference between the pre-computed stats before the split and the stored stats during the split (while holding latches). Fixes: cockroachdb#119516 Release note (ops change): Two new metrics (kv.split.estimated_stats and kv.split.total_bytes_estimates) added to track the number of splits that produce MVCC stats estimates, and the total bytes of estimates produced.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
As a followup to #119499, we should add some metrics around the stats estimates produced by the split.
kv.split.post-pre-split-diff-<stat>: for eachstatin the MVCCStats, the difference between the value of the stat at the time of the split (in splitTrigger) and before the split (in AdminSplit). Not sure if this will be useful/needed for all stats; maybe export some as metrics and log the rest. These will help keep track of drift in specific metrics and allow us to correlate that with other instabilities during investigations.kv.split.fallback-accurate-stats: the number of splits that fall back to accurate splits computation because of a large discrepancy in the stats (batcheval: use a threshold on stats discrepancy to fall back to accurate-stats split #119503).We already log the delta that the consistency checker queue corrects. We expect these deltas to be larger after #119499.
Jira issue: CRDB-36263
Epic CRDB-34215
The text was updated successfully, but these errors were encountered: