Added a collector for Hadoop Metrics

[tehmaze]

Hadoop can generate metric files, containing a time stamp with key-value pairs. All values that are numeric are sent to graphite.

[kormoc]

Thanks!

[kormoc]

When writing the unit test for the collector, it appears that it doesn't process multiple lines per log file very well. If you look at the https://github.com/BrightcoveOS/Diamond/blob/master/src/collectors/HadoopCollector/fixtures/jvmmetrics.log file, there are multiple lines per jvm.metrics key. It appears to me that we want to extract out the hostname and processname and add them to the stats key?

[tehmaze]

Please re-open this issue so I can investigate, it's probaby best to construct a key using the hostName and processName keys, in stead of using the second argument. What do you reckon?

[kormoc]

I don't see a way to reopen the request. We can just start a new issue to track this.

It looks like each style of log has a different set of keys

rpcmetrics.log looks like it would use hostname and port
mrmetrics.log looks like it would only need to use a custom key for mapred.job and it would be hostname, group, and counter
jvmmetrics.log looks like it would use hostname and process name
dfsmetrics.log looks fine as is

Does that seem reasonable?