kvserver: Add a metric for in-progress snapshots

[andrewbaptist]

Fixes: #98242

Knowing how many delegate snapshot requests are currently in-progress will be useful for detecting problems. This change adds a metric for this. It also updates the names of the previous stats to have the prefix range.snapshots vs range.snapshot to be consistent with other stats.

Epic: none

Release note: None

[blathers-crl]

It looks like your PR touches production code but doesn't add or edit any test code. Did you consider adding tests to your PR?

_{🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.}

[cockroach-teamcity]

This change is 

[kvoli]

with release note for renaming failure/success metrics.

Reviewed 2 of 3 files at r1, 1 of 1 files at r2, all commit messages.
Reviewable status: complete! 1 of 0 LGTMs obtained (waiting on @andrewbaptist)

---

-- commits line 9 at r2:
range.snapshots.delegate.failures and range.snapshot.delegate.successes had a release note when they were added (4a43cf6). You should add another release note here mentioning the rename, for docs.

---

pkg/kv/kvserver/replica_command.go line 2859 at r2 (raw file):

		)
		if !selfDelegate {
			r.store.Metrics().DelegateSnapshotInProgress.Dec(1)

nit: Did you consider moving this up above in the same if block as the inc and deferring the dec?

[andrewbaptist]

TFTR - I added the release note.

Reviewable status: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @kvoli)

---

-- commits line 9 at r2:

Previously, kvoli (Austen) wrote…

range.snapshots.delegate.failures and range.snapshot.delegate.successes had a release note when they were added (4a43cf6). You should add another release note here mentioning the rename, for docs.

Done

---

pkg/kv/kvserver/replica_command.go line 2859 at r2 (raw file):

Previously, kvoli (Austen) wrote…

nit: Did you consider moving this up above in the same if block as the inc and deferring the dec?

I did consider this, but then it complains about a defer within a loop. I'm not sure which is better actually but I left this for now.

[kvoli]

Reviewed all commit messages.
Reviewable status: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @andrewbaptist)

[andrewbaptist]

bors r=kvoli

[craig]

Build failed:

• Bazel Essential CI (Cockroach)

[knz]

flake here #100119

bors r=kvoli

[craig]

Build succeeded:

• Bazel Essential CI (Cockroach)

[knz]

Adding the backport label after the PR has merged does nothing. You need to instruct blathers, as follows:

blathers backport 23.1