[TASK] add database type graphite #65

andir · 2017-05-26T10:57:41Z

I've spent the last night hacking on a graphite database backend.

corny · 2017-05-26T11:18:37Z

database/graphite/node.go

+	}
+
+	node_prefix := MeasurementNode + `.` + stats.NodeID + `.` + strings.Replace(nodeinfo.Hostname, ".", "__", -1)
+


Use a small inline function to dry up your code:

addField := func(name string, value interface{}) { fields = append(fields, graphigo.Metric{Name: node_prefix + "." + name, Value: value}) }

Much better 👍

corny · 2017-05-26T11:20:42Z

database/graphite/node.go

+		{Name: node_prefix + ".proc.running", Value: stats.Processes.Running},
+		{Name: node_prefix + ".clients.wifi", Value: stats.Clients.Wifi},
+		{Name: node_prefix + ".clients.wifi24", Value: stats.Clients.Wifi24},
+		{Name: node_prefix + ".clients.wifi5", Value: stats.Clients.Wifi5},


Better use wifi11a and wifi11g in the name.
The old labels wifi24 and wifi5 are deprecated.

corny · 2017-05-27T09:05:50Z

database/graphite/database.go

+						return
+					}
+				} else {
+					defer c.Close()


defer c.Close() vor die Schleife schreiben?

corny · 2017-05-27T09:08:39Z

database/graphite/database.go

+
+func (c *Connection) addWorker() {
+	defer c.wg.Done()
+	for c.Connection != nil {


Hier kannst du einfach über die Punkte iterieren:
for point := range c.points {

corny · 2017-05-27T09:09:27Z

database/graphite/database.go

+}
+
+func (c *Connection) Close() {
+	if c.client.Connection != nil {


hier den Channel schließen close(c.points)

andir · 2017-05-27T11:04:43Z

I still have to run some tests with our local network - waiting on more storage on our graphite node ;-)
I'll report back once that tests was successful

corny · 2017-05-27T11:05:23Z

OK, thanks a lot. Your code looks fine for me now.

corny · 2017-05-27T11:06:28Z

database/graphite/node.go

+	addField("proc.running", stats.Processes.Running)
+	addField("clients.wifi", stats.Clients.Wifi)
+	addField("clients.wifi11a", stats.Clients.Wifi24)
+	addField("clients.wifi11g", stats.Clients.Wifi5)


Maybe just 11a and 11g instead of wifi11a and wifi11g? It implies wifi. Any other opinions?

genofire · 2017-05-27T12:25:54Z

database/graphite/global.go

+	"github.com/fgrosse/graphigo"
+)
+
+func (c *Connection) InserGlobals(stats *runtime.GlobalStats, time time.Time) {


I need a tee in the Morning 😄 : InsertGlobals

genofire · 2017-05-27T12:45:19Z

Without the tipping mistake, it looks good for me as well.
I think we could merge very soon.
(Could i suggest you to write some test for this package? - You could mock to addPoint of the graphite package and run functions of Connection)

coveralls · 2017-05-27T12:58:40Z

Coverage remained the same at 56.506% when pulling 79c0c22 on freifunk-darmstadt:add-graphite into dc24c8b on FreifunkBremen:master.

andir · 2017-05-27T13:07:53Z

I'll try to write a bunch of tests. It probably will be a copy of the influxdb stuff (for the most part)…

genofire · 2017-05-27T17:20:04Z

For me you this PR has a go.
Whould you squash and merge it @corny if @andir say he is ready?

andir · 2017-05-27T18:10:35Z

I've just tried to figure how I can easily mock out that function... Any ideas? I've only seen some ugly hacks or passing the function as a parameter to the real implementation so I can pass the mock in the tests..

andir · 2017-05-28T12:04:39Z

I just realized that yanic is currently dieing whenever the carbon-cache/relay (graphite backend) is being restarted. The influxdb backend seems to have a similar issue - since log.Fatal is used?

Are there any plans to support reconnect?

genofire · 2017-05-28T13:53:36Z

Plans, No. A reconnect was not necessary till now. There is a RFC: #61
But it does not depends on influxdb. It is just the way we handle a lost of connection.

A reconnect would be a restart of yanic by systemd.

andir · 2017-05-30T07:50:16Z

I'll try to get some reconnect code in during the following days.

In the mean time I discovered that often datapoints are missing in graphite (even after raising all the carbon limits): https://stats.test.h4ck.space/dashboard/db/new-dashboard-copy?orgId=1

Any ideas where that comes from? Yanic is set to query nodes every minute. Graphite is configured for 1 datapoint every 60 seconds.

The version statistics look okay but the others are often missing data.

kb-light · 2017-05-30T20:17:38Z

database/graphite/node.go

+	addField("time.idle", int64(stats.Idletime))
+	addField("proc.running", stats.Processes.Running)
+	addField("clients.wifi", stats.Clients.Wifi)
+	addField("clients.wifi11a", stats.Clients.Wifi24)


11a is 5GHz and 11g is 2.4GHz, so they are interchanged. But why not keeping wfi5 and wifi24 as used in database/graphite/global.go and database/influxdb/node.go? Additional 5GHz is not only 11a but even 11n and 11ac and 2.4GHz is even 11b and 11n.

You're right. Then revert it to wifi24 and wifi5.

genofire · 2017-05-31T09:24:19Z

@andir say in IRC, That he is ready with this PR.

The missing Point we think the problem is, how graphite and yanic works.
Explain/example:
Yanic is configuration to collect in 1m.
If yanic was started at 00:03:32 he try to collect every time x:x:32 per multicast and x:x:02 per unicast (if a node did not answer on multicast)

Buy graphite wants to have x:x:0 so a value are dropped or avg, because the response of yanic on a multicast and unicast get the same timeslot in graphite and the previous or next timeslot get empty.

A possible solution configuration graphite = 2x yanic request.

corny · 2017-05-31T20:51:44Z

@andir Does the response of the node really reach yanic? Please add a Printf for debugging or use tcpdump with the node address.

andir · 2017-06-01T08:57:10Z

The wifi metrics are reverted.
I'll try to construct a test case for yanic and the "lost" metrics later today..

coveralls · 2017-06-01T08:58:04Z

Coverage decreased (-0.2%) to 56.272% when pulling 52919a2 on freifunk-darmstadt:add-graphite into dc24c8b on FreifunkBremen:master.

corny · 2017-06-01T20:50:16Z

If yanic was started at 00:03:32 he try to collect every time x:x:32 per multicast and x:x:02 per unicast (if a node did not answer on multicast)

Simply adding a delay before the first run would solve this problem. This should be an seperate issue.

Added graphite backend

8a0215f

corny requested changes May 26, 2017

View reviewed changes

graphite: cleaned up fields logic and renamed wifi metrics

35926ab

corny requested changes May 27, 2017

View reviewed changes

corny reviewed May 27, 2017

View reviewed changes

graphite: smipliefied worker loop, close channel on close

561699e

corny approved these changes May 27, 2017

View reviewed changes

corny reviewed May 27, 2017

View reviewed changes

genofire added the wip label May 27, 2017

genofire changed the title ~~WIP: Added graphite backend~~ [TASK] add database type graphite - WIP May 27, 2017

genofire requested changes May 27, 2017

View reviewed changes

andir added 2 commits May 27, 2017 14:56

graphite: Fixed typo in InsertGlobals (was InserGlobals)

792f904

graphite: Initialize points channel

79c0c22

genofire approved these changes May 27, 2017

View reviewed changes

kb-light reviewed May 30, 2017

View reviewed changes

genofire changed the title ~~[TASK] add database type graphite - WIP~~ [TASK] add database type graphite May 31, 2017

genofire added review and removed wip labels May 31, 2017

graphite: reversted back to wifi24 and wifi5 metric names

52919a2

corny merged commit 88975d2 into FreifunkBremen:master Jun 1, 2017

corny mentioned this pull request Jun 1, 2017

Delay startup until a multiple of the period since zero time #68

Merged

andir mentioned this pull request Jun 16, 2017

Monitoring im Freifunk (yanic) FreifunkBremen/bremen.freifunk.net#34

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[TASK] add database type graphite #65

[TASK] add database type graphite #65

andir commented May 26, 2017

corny May 26, 2017

andir May 26, 2017

corny May 26, 2017

corny May 26, 2017

corny May 27, 2017

corny May 27, 2017

corny May 27, 2017

andir commented May 27, 2017

corny commented May 27, 2017

corny May 27, 2017

genofire May 27, 2017

genofire commented May 27, 2017

coveralls commented May 27, 2017 •

edited

Loading

andir commented May 27, 2017

genofire commented May 27, 2017

andir commented May 27, 2017

andir commented May 28, 2017

genofire commented May 28, 2017

andir commented May 30, 2017

kb-light May 30, 2017

corny May 31, 2017

genofire commented May 31, 2017 •

edited

Loading

corny commented May 31, 2017

andir commented Jun 1, 2017

coveralls commented Jun 1, 2017 •

edited

Loading

corny commented Jun 1, 2017 •

edited

Loading

		}

		node_prefix := MeasurementNode + `.` + stats.NodeID + `.` + strings.Replace(nodeinfo.Hostname, ".", "__", -1)

[TASK] add database type graphite #65

[TASK] add database type graphite #65

Conversation

andir commented May 26, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andir commented May 27, 2017

corny commented May 27, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

genofire commented May 27, 2017

coveralls commented May 27, 2017 • edited Loading

andir commented May 27, 2017

genofire commented May 27, 2017

andir commented May 27, 2017

andir commented May 28, 2017

genofire commented May 28, 2017

andir commented May 30, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

genofire commented May 31, 2017 • edited Loading

corny commented May 31, 2017

andir commented Jun 1, 2017

coveralls commented Jun 1, 2017 • edited Loading

corny commented Jun 1, 2017 • edited Loading

coveralls commented May 27, 2017 •

edited

Loading

genofire commented May 31, 2017 •

edited

Loading

coveralls commented Jun 1, 2017 •

edited

Loading

corny commented Jun 1, 2017 •

edited

Loading