vttablet/tabletmanager: add isInSRVKeyspace/isShardServing #7929
Conversation
Signed-off-by: Guido Iaquinti <giaquinti@slack-corp.com>
| @@ -115,6 +118,7 @@ func init() { | |||
| statsTabletType = stats.NewString("TabletType") | |||
| statsTabletTypeCount = stats.NewCountersWithSingleLabel("TabletTypeCount", "Number of times the tablet changed to the labeled type", "type") | |||
| statsBackupIsRunning = stats.NewGaugesWithMultiLabels("BackupIsRunning", "Whether a backup is running", []string{"mode"}) | |||
| statsIsInSRVKeyspace = stats.NewGauge("IsInSRVKeyspace", "Whether the vttablet is in the serving keyspace (1 = true / 0 = false)") | |||
There was a problem hiding this comment.
Should be IsInSrvKeyspace otherwise the prometheus metric might turn out to be vttablet_is_in_s_r_v_keyspace
There was a problem hiding this comment.
Should be
IsInSrvKeyspaceotherwise the prometheus metric might turn out to bevttablet_is_in_s_r_v_keyspace
Actually I think it is rendered correctly:
# HELP vttablet_is_in_srv_keyspace Whether the vttablet is in the serving keyspace (1 = true / 0 = false)
# TYPE vttablet_is_in_srv_keyspace gauge
vttablet_is_in_srv_keyspace 0
There was a problem hiding this comment.
TIL :)
It will still be nice to rename it to be consistent with how we use SrvKeyspace everywhere else in the codebase.
|
I found out that with this implementation the topology info is not propagated to the tablet manager state after operations like |
Signed-off-by: Guido Iaquinti <giaquinti@slack-corp.com>
I think I found the underlying issue: in few places of the codebase we call I think the right approach here to don't end up with stale info in the tablet manager state is to make sure we call |
| @@ -154,6 +155,158 @@ func TestStateTabletControls(t *testing.T) { | |||
| assert.False(t, qsc.IsServing()) | |||
| } | |||
|
|
|||
| func TestStateIsShardServingisInSrvKeyspace(t *testing.T) { | |||
There was a problem hiding this comment.
If we want to be super throughout with the testing, we should also assert that the gauge ends up with the expected values when we call func (ts *tmState) Open(). That's one more place where updateLocked will be called and we expect the gauge to be updated.
Adding this test, will make sure that we will detect a regression if the Open method is refactored and no longer calls updateLocked.
There was a problem hiding this comment.
Make sense, let me add that. Thanks!
Signed-off-by: Guido Iaquinti giaquinti@slack-corp.com
Description
Add a new metric in
vttabletto keep track if the keyspace partition it belongs to is serving or not. This metric can be useful for external systems to automatically enable/disable alerts.In order to don't add additional watches to the topology service (as we don't want those to linearly scale with the number of vttablets), we refresh the serving keyspace view in TabletManager only when
RefreshFromTopoInfois called.Note: this PR is still a WIP but I'll appreciate an early feedback.
Related Issue(s)
N/A
Checklist
Deployment Notes
N/A
Impacted Areas in Vitess
Components that this PR will affect: