Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Graph to show server restarts over time #712

Closed
noamha opened this issue Aug 7, 2019 · 10 comments · Fixed by #738
Closed

Graph to show server restarts over time #712

noamha opened this issue Aug 7, 2019 · 10 comments · Fixed by #738

Comments

@noamha
Copy link

@noamha noamha commented Aug 7, 2019

Today if you want to have an indication if a server went through a restart over a big period of time (for example two weeks) it's hard to see it.

An easy indication would be to have a graph of scylla_gossip_heart_beat. Then you would see a big drop and rise in the number and it would outstand over large periods.

@dorlaor

This comment has been minimized.

Copy link

@dorlaor dorlaor commented Aug 7, 2019

@tzach

This comment has been minimized.

Copy link
Contributor

@tzach tzach commented Aug 7, 2019

@amnonh

This comment has been minimized.

Copy link
Contributor

@amnonh amnonh commented Aug 18, 2019

A preview of the suggested panel
image

@amnonh amnonh added this to the monitoring 3.1 milestone Aug 18, 2019
@tzach

This comment has been minimized.

Copy link
Contributor

@tzach tzach commented Aug 18, 2019

@amnonh using a line chart looks weird here. It makes it look like there were 0.5 restarts at some point in time. Also, the Y-axis is basically 0 or 1
A bar graph will be better since it does not show 1/2 value.
The best will be to use annotation like markers, or status map

@amnonh

This comment has been minimized.

Copy link
Contributor

@amnonh amnonh commented Aug 18, 2019

As you can see the legend is in integer numbers, values can be more than one if you group by DC or cluster.

This is how it looks with bar graph (accumulative to support multiple restarts)
image

@noamha

This comment has been minimized.

Copy link
Author

@noamha noamha commented Aug 18, 2019

@amnonh

This comment has been minimized.

Copy link
Contributor

@amnonh amnonh commented Aug 18, 2019

This is an example using annotations:
image

@tzach

This comment has been minimized.

Copy link
Contributor

@tzach tzach commented Aug 18, 2019

Can you add text to the annotation, as: "node 1.2.3.4 restart" ?

@amnonh

This comment has been minimized.

Copy link
Contributor

@amnonh amnonh commented Aug 18, 2019

There is a hover, and it looks like:
Screenshot from 2019-08-18 15-34-46

@tzach

This comment has been minimized.

Copy link
Contributor

@tzach tzach commented Aug 18, 2019

Adding such annotation on the cluster view dashboard will be super useful

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.