Multiple Collector instances

[pablolb]

Hi,

Today I was trying to setup Diamond to collect data from several servers, each containing multiple instances of memcached.

I cannot see how to accomplish this right now, as the collectors are added to a dict with their class name as key.

I have created a branch in my fork with a solution: https://github.com/pablolb/Diamond/tree/multiple-collector-instances.

But maybe you have a workaround or you'd prefer a better way to solve this...?

In the first commit (pablolb@f2397aa) I scan collectors config directory and schedule new instances of the collector.

In the second commit, I use MemcachedCollector as an example as how this can be used... I've seen this used in Genttoo network scripts I think (using symlinks and interpreting the actual name)

pablo@localhost collectors $ ls -l
total 4
-rw-r--r-- 1 root root 61 Mar  5 22:40 MemcachedCollector.conf
lrwxrwxrwx 1 root root 23 Mar  5 22:40 MemcachedCollector-localhost:11212.conf -> MemcachedCollector.conf

pablo@localhost collectors $ cat MemcachedCollector.conf

enabled=True
override_hostname=True
custom_name_is_host=True

[kormoc]

Yes, Gentoo uses that way for it's init.d network scripts

That said, I think the preferred solution would be to setup the config file to use subkeys

Example

[MemcachedCollector]

[[Identifier A]]
host = localhost
port = 11211

[[Identifier B]]
host = localhost
port = 22122

I feel that would be much easier for systems like bcfg2 or chef to manage.

Would that work for your usage?

[pablolb]

Yes, this would work fine.

I browsed over the current collectors, and I saw that at leas one uses sub-sections in their configuration.

Maybe a special named section would be required for named instances? I have pushed a commit to the same branch as an example.

[kormoc]

I think this should do the trick really well. Please let me know if you run into any issues at all.

Thanks!

[keen99]

just a note for future readers who are looking for something like this and find this in a search:

this is NOT supported syntax for current (june 2014) diamond.

:-)

[kovyrin]

So, do I understand it correctly: there is no way to monitor more than one instance of a service unless the author of a collector invented a special syntax for defining multiple sources of metrics? :-(

[kormoc]

As of now, that is correct. A PR that implements a generic solution would be welcome