kvserver,server: add more load stats to hot ranges v2 api #95388
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
This change is |
kvoli
changed the title
kvoli
force-pushed
the
230117.hotranges-stats-v2
branch
from
d26f769
to
5e5d197
Compare
|
It looks like your PR touches production code but doesn't add or edit any test code. Did you consider adding tests to your PR? 🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf. |
kvoli
force-pushed
the
230117.hotranges-stats-v2
branch
2 times, most recently
from
6bfc576
to
c109235
Compare
kvoli
changed the title
kvoli
force-pushed
the
230117.hotranges-stats-v2
branch
from
c109235
to
82f9c90
Compare
kvoli
changed the title
Various range load statistics such as read(write) bytes(keys) exist on the hot ranges (v1) and range statistics api. This commit adds the additional statistic, `CPUTimePerSecond`. This statistic is the recent average cpu usage per second of the range in nanoseconds. resolves: cockroachdb#95384 Release note: None
kvoli
force-pushed
the
230117.hotranges-stats-v2
branch
from
82f9c90
to
718c70a
Compare
koorosh
reviewed
Jan 24, 2023
There was a problem hiding this comment.
LGTM with only one minor suggestion.
Reviewed 2 of 4 files at r1, 9 of 9 files at r2, 6 of 6 files at r3, all commit messages.
Reviewable status:complete! 0 of 0 LGTMs obtained (waiting on @kvoli)
pkg/server/serverpb/status.proto line 1415 at r3 (raw file):
(gogoproto.customname) = "QPS" ]; // writes_per_second is the recent number of keys written per second on
nit. 11 is the next vacant field number available to use and suggesting to put fields in order.
kvoli
force-pushed
the
230117.hotranges-stats-v2
branch
from
718c70a
to
24d61d1
Compare
kvoli
commented
Jan 24, 2023
There was a problem hiding this comment.
Reviewable status:
pkg/server/serverpb/status.proto line 1415 at r3 (raw file):
Previously, koorosh (Andrii Vorobiov) wrote…
nit. 11 is the next vacant field number available to use and suggesting to put fields in order.
Updated.
Previously, only QPS was included in the hot ranges v2 api. This commit adds the following range statistics to the API: ``` WritesPerSecond ReadsPerSecond WriteBytesPerSecond ReadBytesPerSecond CPUTimePerSecond ``` See the associated comments in `status.proto` for a description of each load statistic. These may be used to determine the hottest ranges. Note however that these are sorted by default by QPS. The ordering of ranges w.r.t the new load statistics has no guarantees. Likewise, ranges may be omitted which have high relative values for the new load statistics but relatively low QPS. In practice however, there usually exists a strong correlation that ensures that such ranges would not be omitted entirely. resolves: cockroachdb#95383 Release note: None
kvoli
force-pushed
the
230117.hotranges-stats-v2
branch
from
24d61d1
to
134463b
Compare
koorosh
approved these changes
Jan 25, 2023
There was a problem hiding this comment.
Reviewed 1 of 1 files at r4, 1 of 1 files at r5, all commit messages.
Reviewable status:
|
TYFTR - merging in #95190 |
craig bot
pushed a commit
that referenced
this pull request
Jan 27, 2023
95190: ui: display more load stats on /hotranges page r=koorosh a=kvoli First two commits are from #95388. Previously the hot ranges page only showed one statistic that helped identify high load ("hot") ranges: queries per second (QPS). QPS is a measure of the number of batch request a replica processed per second, averaged over the last 30 minutes. If the batch request composition is non-uniform in terms of incidental load on the cluster, the correlation between QPS and importance of a range to the end user weakens. This commit adds more statistics per-range to the hot ranges ui page to provide better insight into load composition. The statistics are rated (per-second) and averaged over the last 30 minutes, the same as the existing load statistic QPS. - CPU: CPU time used in processing this range. - Write (keys): number of keys written on this range. - Write (bytes): number of bytes written on this range. - Read (keys): number of keys read on this range. - Read (bytes): number of bytes read on this range. The ranges that are displayed on the hot ranges page can be ordered using these statistics as a key. An example page view is shown below running kv50.  depends on: #95388 resolves: #95386 Release note (ui change) Add write bytes, write keys, read bytes, read keys and cpu statistics to the `/hotranges` db-console page. These statistics are the rated average over the last 30 minutes. Co-authored-by: Austen McClernon <austen@cockroachlabs.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds CPU as a hotranges v1 and range statistic field. Then the stats that are included in the v1 hot ranges (including cpu) but not v2 are added to the hot ranges v2 api.
See the associated comments in
status.protofor a description of eachload statistic.
These may be used to determine the hottest ranges. Note however that
these are sorted by default by QPS. The ordering of ranges w.r.t the new
load statistics has no guarantees. Likewise, ranges may be omitted which
have high relative values for the new load statistics but relatively low
QPS. In practice however, there usually exists a strong correlation that
ensures that such ranges would not be omitted entirely.
resolves: #95383
resolves: #95384
Release note: None