Metricbeat: Galera module and status Metricset #6892

xy795 · 2018-04-18T10:01:24Z

Add module and metricset for collecting metrics from a Galera-MySQL-Cluster-Node.

Features:

Uses MySQL's "show status" command to fetch data from a node.
Has two config options to specify the range of the collected data (small/full).
Documented field descriptions (see Galera docs for comparison).
Config and structure otherwise very similar to the "mysql" module.

Other changes in metricbeat: Moved govendor dependency from the "mysql" module to metribeat root, cause it's needed in the "galera" module as well.

Other: Added python virtualenv parameter to use python2 by default because it's needed and caused me various issues and much frustration.

Please let me now if you think I missed something in this module or if something needs further improvements. Otherwise I would love to see this module in the official beats repository!

update

through the time and effort it would take to make this possible in a Docker container (not to mention it would be not very esthetic). Also this module is nearly the same as the mysql module with the only difference beeing that it queries only galera-specific fields (wsrep_ prefix) and so matching those fields for the event. Added virtualenv param for python2 cause this is needed and (for some reason) not specified by default.

elasticmachine · 2018-04-18T10:01:26Z

Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually?

houndci-bot · 2018-04-18T10:01:35Z

metricbeat/module/galera/status/data.go

+	}
+
+	// Schema for mapping Galera-Status-Variables related to cluster health
+	schema_small = s.Schema{


don't use underscores in Go names; var schema_small should be schemaSmall

houndci-bot · 2018-04-18T10:01:36Z

metricbeat/module/galera/status/data.go

+
+var (
+	// Schema for mapping all Galera-Status-Variables
+	schema_full = s.Schema{


don't use underscores in Go names; var schema_full should be schemaFull

houndci-bot · 2018-04-18T10:01:36Z

metricbeat/module/galera/galera.go

+/*
+Package galera is Metricbeat module for Galera Cluster.
+*/
+


package comment is detached; there should be no blank lines between it and the package statement

ruflin

Thanks for the contribution. Only skimmed through it so far and left some high level comments?

How can we test this? Normally we have a Dockerfile for each service, not sure if this is possible for Galera?

Do you have a link to the full list of metrics this is gathering?

ruflin · 2018-04-18T13:52:43Z

NOTICE.txt

@@ -630,9 +630,9 @@ THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR I

 --------------------------------------------------------------------
 Dependency: github.com/go-sql-driver/mysql
-Revision: 9dee4ca50b83acdf57a35fb9e6fb4be640afa2f3
+Revision: cd4cb909ce1a31435164be29bf3682031f61539a


Could you open a separate PR to update the mysql dependency? I think we should do that anyway.

ruflin · 2018-04-18T13:52:49Z

libbeat/scripts/Makefile

@@ -68,7 +68,7 @@ GOPACKAGES_COMMA_SEP=$(subst $(space),$(comma),$(strip ${GOPACKAGES}))
 PYTHON_ENV?=${BUILD_DIR}/python-env
 PIP_INSTALL_PARAMS?=
 BUILDID?=$(shell git rev-parse HEAD) ## @Building The build ID
-VIRTUALENV_PARAMS?=
+VIRTUALENV_PARAMS?=-p /usr/bin/python2


ruflin · 2018-04-18T13:55:37Z

metricbeat/module/galera/_meta/config.yml

+  enabled: false
+  period: 10s
+
+  # Specifies the status variables to query from the cluster


The short config should not contain all the config options. Please also create a config.reference.yml file.

ruflin · 2018-04-18T13:56:11Z

metricbeat/module/galera/_meta/evn

@@ -0,0 +1,3 @@
+GALERA_DSN=root:test@tcp(galera:3306)/


should the name of the file be env?

ruflin · 2018-04-18T13:57:15Z

Makefile

@@ -5,7 +5,7 @@ PROJECTS=libbeat $(BEATS)
 PROJECTS_ENV=libbeat filebeat metricbeat
 SNAPSHOT?=yes
 PYTHON_ENV?=$(BUILD_DIR)/python-env
-VIRTUALENV_PARAMS?=
+VIRTUALENV_PARAMS?=-p /usr/var/python2


We should not change the defaults here as it will potentially break other things. Perhaps you can open a separate issue / PR to discuss this in more detail?

This seems to break travis build

exekias · 2018-04-18T16:47:57Z

Thank your contribution, the module is looking really good!! Could you please sign the CLA? Also, it will need a PR as it's conflicting with master

exekias · 2018-04-18T16:49:19Z

metricbeat/module/galera/status/status.go

+
+// New create a new instance of the MetricSet
+// Loads query_mode config setting from the config file
+func New(base mb.BaseMetricSet) (mb.MetricSet, error) {


I think we should flag this as experimental for the first version, could you add something similar to this?

beats/metricbeat/module/ceph/pool_disk/pool_disk.go

Line 36 in 746f8b9

cfgwarn.Experimental("The ceph pool_disk metricset is experimental")

exekias · 2018-04-18T16:50:45Z

Thank your contribution, the module is looking really good!! Could you please sign the CLA? Also, it will need a PR as it's conflicting with master.

Also, do you happen to have a dashboard for this module? It would be really nice to add it here 😇 . If you are willing to, you can have a look to docs on how to do it https://www.elastic.co/guide/en/beats/devguide/current/new-dashboards.html

jsoriano · 2018-04-19T09:11:00Z

Awesome contribution 🙂

I think it could fit as a metricset of mysql module, or even as an optionaly-enabled part of the mysql status metricset.

jsoriano · 2018-04-19T10:50:38Z

@xy795 we have talked about this offline, and we think this should be included as a metricset of the mysql module, as all the connection and query logic could be reused, wdyt? Please tell me if you see any problem moving this to a mysql metricset.

added galera-status metricset to the mysql module moved the vendor go-sql-drivers folder back to the mysql module

houndci-bot · 2018-04-25T10:53:48Z

metricbeat/module/mysql/galera-status/status.go

+http://dev.mysql.com/doc/refman/5.7/en/show-status.html
+http://galeracluster.com/documentation-webpages/galerastatusvariables.html
+*/
+package galeraStatus


don't use MixedCaps in package name; galeraStatus should be galerastatus

houndci-bot · 2018-04-25T10:53:48Z

metricbeat/module/mysql/galera-status/status.go

@@ -0,0 +1,116 @@
+/*


package comment should be of the form "Package galeraStatus ..."

houndci-bot · 2018-04-25T10:53:48Z

metricbeat/module/mysql/galera-status/data.go

@@ -0,0 +1,144 @@
+package galeraStatus


don't use MixedCaps in package name; galeraStatus should be galerastatus

houndci-bot · 2018-04-25T10:57:05Z

metricbeat/module/mysql/galera-status/status.go

@@ -0,0 +1,116 @@
+/*


package comment should be of the form "Package galerastatus ..."

xy795 · 2018-04-25T11:21:31Z

@jsoriano you are absolutely right, I thought about this earlier, but for some reason I decided to make an own module. Now I moved the metricset to the mysql module, since there is no reason to make a own module for galera.

@exekias I think we signed the CLA already. You should be able to find it under company "F24 IT-Services GmbH"

@ruflin I know there are no tests for this. I tried for several days setting up a Docker test for this, but did not manage to get the tests working with this. I tested the metricset very well with a local Galera-Cluster and found no errors so far. Since this metricset does nearly the same thing as the official mysql/status set I decided it would not be reasonable to invest that much time in creating a functioning test for this. I hope this is acceptable for you. Otherwise do you have an other idea to make tests in this case possible?

jsoriano · 2018-04-25T11:25:15Z

metricbeat/module/mysql/galera-status/data.go

+		},
+		"flow_ctl": s.Object{
+			"paused": c.Float("wsrep_flow_control_paused"),
+			"paused_ns": c.Int("wsrep_flow_control_paused_ns"),


These metrics are quite important in galera, they probably deserve to be also in the small metric set 🙂

jsoriano

Thanks for addressing this as a new metricset! I have added some things, the only important one would be to mark it as experimental by now.

jsoriano · 2018-04-25T11:28:42Z

metricbeat/module/mysql/galera-status/status.go

+
+// New create a new instance of the MetricSet
+// Loads query_mode config setting from the config file
+func New(base mb.BaseMetricSet) (mb.MetricSet, error) {


Please add here a line like this one:

cfgwarn.Experimental("The mysql galera-status metricset is experimental.")

I did this already.

jsoriano · 2018-04-25T11:31:42Z

metricbeat/module/mysql/galera-status/status.go

+}
+
+// loadStatus loads all status entries from the given database into an array.
+func (m *MetricSet) loadStatus(db *sql.DB) (map[string]string, error) {


This method is quite similar in the status metricset, we could probably reuse some code. We can leave it for a future PR.

Indeed most of the code in fetch and event mapping can also be reused.

jsoriano · 2018-04-25T11:37:48Z

metricbeat/module/mysql/galera-status/_meta/fields.yml

+- name: status
+  type: group
+  description: >
+    `status` contains the metrics that were obtained by the status SQL-Galera query.


Add here also the experimental mark with:

release: experimental

jsoriano · 2018-05-16T15:00:24Z

@xy795 could you please update the branch and run make update again?

xy795 · 2018-05-19T20:37:26Z

@jsoriano when I run make update no tracked files are changed.
What do you mean by updating the branch?

jsoriano · 2018-05-21T11:32:51Z

With updating the branch I mean to merge or rebase it with origin/master so it gets updated with the master branch.
The tests complain about not updated files, in principle make update should fix this.

jsoriano · 2018-05-23T09:53:13Z

Thanks @xy795, this looks better now, the tests failing don't seem related to this change :)

jsoriano · 2018-05-23T09:54:57Z

jenkins, test this please

jsoriano · 2018-05-23T09:58:55Z

@xy795 but there are still issues with the CLA :( could you please check that you have signed it with the same email address that you are using for your commits?

jsoriano · 2018-05-23T13:11:39Z

jenkins, test this again, please

xy795 · 2018-06-03T09:45:06Z

@jsoriano the CLA should now be complete

jsoriano · 2018-06-03T10:04:16Z

@xy795 sorry, but it still doesn't appear as signed :( are you signing it with the same mail address that the one in your commits?

xy795 · 2018-06-13T10:33:40Z

@jsoriano I checked it again and now it schoul'd finally all be fine.

jsoriano · 2018-06-13T11:15:52Z

@xy795 I'm sorry but we cannot find any signature with your username, or with the name or email in the commits of this PR (author appears as Adrian Marquis, with a mail address from f24.com).
Could you try to repeat the process of signing the agreement here? https://www.elastic.co/contributor-agreement
Or if you signed with other email and/or name and you want to use this one, you can rewrite the commits history.

xy795 · 2018-06-13T21:08:26Z

@jsoriano I rechecked all my commits and all of them has the correct author (the one that is in the CLA which was signed today again)

ruflin · 2018-06-14T11:34:15Z

@xy795 @jsoriano As it's a company CLA it works a bit different and some manual steps are needed. Now everything should be fine.

PR will need an other rebase.

jsoriano · 2018-06-15T10:27:39Z

Thanks @ruflin

ruflin · 2018-06-19T11:08:11Z

@xy795 Any chance to get an other rebase and make update for this one to get it in?

# Conflicts: # metricbeat/include/fields.go

jsoriano · 2018-06-20T16:39:01Z

@xy795 fields need to be updated after updating with master

jsoriano · 2018-06-26T08:34:34Z

jenkins, test this please

jsoriano · 2018-06-26T09:18:39Z

Failing test doesn't seem related, I'm merging this.

jsoriano · 2018-06-26T09:20:20Z

Thanks @xy795 for the effort here!

Adrian Marquis added 6 commits January 11, 2018 16:54

Added galera module

1dfb71a

Some tests for galera module

fddf09f

Merge remote-tracking branch 'upstream/master'

df7adff

update

Added galera assets

18d83e8

Added govendor entry for go-sql-driver to metricbeat/vendor

7bd94ee

houndci-bot reviewed Apr 18, 2018

View reviewed changes

Resolved code-style violations

313ab2c

ruflin reviewed Apr 18, 2018

View reviewed changes

ruflin added module review Metricbeat Metricbeat labels Apr 18, 2018

exekias reviewed Apr 18, 2018

View reviewed changes

deleted galera module

ac66f54

added galera-status metricset to the mysql module moved the vendor go-sql-drivers folder back to the mysql module

houndci-bot reviewed Apr 25, 2018

View reviewed changes

satisfy houndbot

3a219f5

houndci-bot reviewed Apr 25, 2018

View reviewed changes

Adrian Marquis added 2 commits April 25, 2018 12:58

whoops, again

a6da320

Added experimental config warning

227a164

Removed the python virtualenv option in the Makefiles

4a775be

jsoriano reviewed Apr 25, 2018

View reviewed changes

Extended small set

877598b

jsoriano requested changes Apr 25, 2018

View reviewed changes

Add experimental flag

2a6cbee

Adrian Marquis added 2 commits May 16, 2018 10:58

Fixed Travis build

de5a0b9

Totally overseen this

15a1487

Adrian Marquis added 2 commits May 23, 2018 10:12

Merge branch 'master' of https://github.com/elastic/beats

5556efe

Updated docs

b832a0a

jsoriano approved these changes May 23, 2018

View reviewed changes

Adrian Marquis added 2 commits June 20, 2018 10:30

Updated fields

241cb2e

Merge remote-tracking branch 'remotes/upstream/master'

c508b1c

# Conflicts: # metricbeat/include/fields.go

Updated fields

3029cfc

jsoriano merged commit f29fa39 into elastic:master Jun 26, 2018

jsoriano mentioned this pull request Jun 27, 2018

Fix naming of fields for galera status #7435

Merged

ph mentioned this pull request Jun 27, 2018

Missing headers on the mysql/galera_status #7450

Merged

Metricbeat: Galera module and status Metricset #6892

Metricbeat: Galera module and status Metricset #6892

Conversation

xy795 commented Apr 18, 2018

elasticmachine commented Apr 18, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ruflin left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

exekias commented Apr 18, 2018

Choose a reason for hiding this comment

exekias commented Apr 18, 2018

jsoriano commented Apr 19, 2018

jsoriano commented Apr 19, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

xy795 commented Apr 25, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jsoriano left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jsoriano commented May 16, 2018

xy795 commented May 19, 2018

jsoriano commented May 21, 2018

jsoriano commented May 23, 2018

jsoriano commented May 23, 2018

jsoriano commented May 23, 2018

jsoriano commented May 23, 2018

xy795 commented Jun 3, 2018

jsoriano commented Jun 3, 2018

xy795 commented Jun 13, 2018

jsoriano commented Jun 13, 2018

xy795 commented Jun 13, 2018

ruflin commented Jun 14, 2018

jsoriano commented Jun 15, 2018

ruflin commented Jun 19, 2018

jsoriano commented Jun 20, 2018

jsoriano commented Jun 26, 2018

jsoriano commented Jun 26, 2018

jsoriano commented Jun 26, 2018