Consider public, read-only Grafana dashboards

[nkcmr]

Hi! 👋 First of all, thank you for running hachyderm.io. It runs very smoothly compared to other instances!

To the point: I've noticed @krisnova posting some screenshots of Grafana dashboards. Maybe there could be a small collection of publicly-accessible dashboards that show critical metrics.

Pros:

• Allows crowd-source issue detection
  Cons:
• Might turn into a thing that people nag the admins/mods with? (example: "This dashboard is going up and to the right! That seems bad!", turns out it is fine)

Thanks again!

[kingdonb]

I just wanted to chime in and echo that the transparency is awesome, I think I saw the toot you're talking about in the server-wide notification, I was wondering if there's a way to go back and read old notifications that came that way from the instance?

I tooted about it here: https://hachyderm.io/web/@yebyen/109303826183477956

I think public dashboards would be helpful for anyone who is going to wear a pager for Hachyderm, at least I think that those dashboards would be helpful, whether or not they are public. I am in general interested in the observability of Hachyderm and how things are going as a service, and if I'm able to help in any way I'm more than interested.

My experience is with kube-prometheus-stack, for now

I don't know well enough or in any way even where Hachyderm things are deployed for myself to say how they should be monitored or what metrics to expose, at this point, but I'm interested and would be glad to help answer this by spending some volunteer time, or starting up a stream and/or tuning in and listening at a particular time... you know, in order to receive the benefit of the community's instructions, if that's something we'd like or if that type of thing was going to be a thing...

[dmah42]

i would be ok with exposing some of these grafana dashboards as read-only... i have no idea if this is feasible (https://grafana.com/docs/grafana/latest/dashboards/dashboard-public/ suggests it's an alpha opt-in feature).

[dmah42]

starting to figure out which charts we'd want to show.

but statsd export from Mastodon doesn't containing other interesting things like active users, posts per day, etc. we'd want to pull that from the API and would need a custom prometheus exporter.

maybe, given the interest in mastodon, a prometheus-mastodon-exporter isn't a terrible idea.. it could pull from statsd and the API and power some interesting dashboards.

[kingdonb]

The link that was posted in the Announcements board is excellent:

These are the metrics that I imagine people will be most interested in. Is it OK to post the dashboard link here itself, or should we wait to publish it until there is a stable URL? (I screencapped it, which should be enough to help us gather feedback and to facilitate the iteration... the URL is a numbered hash though, and I don't want to create bad links in a permanent/public setting in case it won't be stable or permanent... so I haven't linked it here.)

[dmah42]

i think that link is pretty stable (if ugly)

[nkcmr]

This issue feels closable to me. Those metrics are great and provide exactly what I was looking for :)

I'll let the mods/admins decide if it should be closed.

[dmah42]

we will keep adding more, but I agree.