add example decoder for linux /proc/stat

[steverweber]

/proc/stat is a good source for cpu utilization metric. Where /proc/loadavg is kinda tricky to work with.
/proc/stat can give 1 second resolution without any issues.

however getting the information is not so simple.
You have to pull /proc/stat twice to solve the delta of the values to gauge preformance.

This is my first time playing with lua lpeg and im still vary green to the heka project.

This pull request is just to get some thoughs on how a good heka plugin could be created for the /proc/stat cpu metric.

This solution is not ideal! but it's giving me good data.

[stat_ProcessInput]
type = "ProcessInput"
decoder = "StatDecoder"
ticker_interval = 3
stdout = true
stderr = false
    [stat_ProcessInput.command.0]
    bin = "/bin/sh"
    args = ["-c",'A=`head -1 /proc/stat`; sleep 1; B=`head -1 /proc/stat`; echo ${A}zzz${B}zzz;']

{#  This would be best ...  but I dont see how I can get the diff of the previous read...
    [stat]
    type = "FilePollingInput"
    ticker_interval = 1
    file_path = "/proc/stat"
    decoder = "StatDecoder"
#}

[StatDecoder]
type = "SandboxDecoder"
filename = "lua_decoders/linux_stat.lua"

[chancez]

Use a filter to compute the deltas. Take a look here for an example:

https://github.com/mozilla-services/heka/blob/dev/sandbox/lua/filters/diskstats.lua

[trink]

Not sure what the state of this is but Ben was already working on it.
https://gist.github.com/bbangert/a75405fe9c805c566cf4

Trink

On Sun, Feb 1, 2015 at 8:51 PM, Chance Zibolski notifications@github.com
wrote:

Use a filter to compute the deltas. Take a look here for an example:

https://github.com/mozilla-services/heka/blob/dev/sandbox/lua/filters/diskstats.lua

—
Reply to this email directly or view it on GitHub
#1301 (comment)
.

[steverweber]

thanks!
Ben's solution also needs work... Is he activaly working on it?
@bbangert

[bbangert]

I've been using the procstat filter as the gist shows. It gets the information we need so that was as far as I took it. I assume by 'needs work' you're referring to extracting some of the other data in there?

[steverweber]

@bbangert Honestly I havent took the time to grok your code yet... at first glance some of the docs could use cleanup and the 'for 100 loop' should be redone. Im on a system with 64cpus in the list I would not be supprised if others have servers with > 100.

Also it would be nice if some configurations could be added to ignore data that is not needed.
Personally im only interested in the global cpu stat delta I dont need to proccess the other cpu[0-63].

[steverweber]

@bbangert Your code does look good tho :) do you have any plans on improving it?

[bbangert]

@steverweber Not at the moment. The main reason I wanted per-cpu was to try and determine if our daemon wasn't fully utilizing cores equally (or failing to use a few entirely). This works fine for our use-case, so I have no plan to return to it right now.

[steverweber]

@bbangert interesting usecase.. Since this code is not a priority of yours I hope you dont mind if I fork it and hopfully get it merged into heka.

[steverweber]

i'll likly add a few tweaks...

[bbangert]

@steverweber sure, sounds good

[steverweber]

this is depericated by #1320