Inbuilt GlusterFlow v2 exporter

[justinclift]

Do you reckon it would be much extra work to add a (hopefully simple) exporter for GlusterFlow v2? (to be built using InfluxDB, which is also go 😉)

All it would need to do is send the details of each client request to an InfluxDB database. (from each of the GlusterD daemons) Without knowing better, that doesn't sound like a huge amount of code. 😀

The benefit of that (especially for development) is being able to diagnose/understand (and maybe replay?) exactly what operations occurred from each member of the cluster, in which order. With accurate timestamps.

[kshlm]

Currently clients just connect to GlusterD to get the volfiles. Is this the information you are looking for? (I may have misunderstood what you meant)
We plan to have APIs to expose a lot of the internal information in GlusterD to the outside world for consumption. Your request would become a part of that.

[kshlm]

Did you mean the connection and requests between GlusterDs?

[justinclift]

Yeah, I'm just sort of wondering if there is a good way to capture client requests (or maybe server-server GlusterD requests too) into a central time-series database for later analysis.

For client requests, that can expose things like the heaviest used files in a volume at any given time. For server-server processes, it could be used to understand what the request patterns are like in real world operation. (instead of what they "should be" doing in theory)

Kind of thinking and wondering out loud. 😀

[kshlm]

This is a future feature that we will try to enable for GlusterD-2.0.

[justinclift]

Cool. 😄

[aravindavk]

We started adding gluster metrics to https://github.com/gluster/gluster-prometheus project. Closing this issue since we are not working on adding metrics to other collectors now. Please re-open the issue if this feature is important to be fixed.

[justinclift]

Looking at the metrics exported by that Prometheus exporter: https://github.com/gluster/gluster-prometheus/blob/master/docs/metrics.adoc They're completely different to the type of metrics useful for a GlusterFlow approach. Also, it's a bit hard to re-open this issue when "Issues" have been turned off completely for the repo. 😉

[aravindavk]

Ok, I will keep this issue open. Thanks