Import data directly rather than via MMV

[lzap]

There is an API to write directly to logs: http://pcp.io/docs/lab.importdata.html

This sounds better than sampling, but there are no golang bindings. Perl prototype could be easy tho, but this should be rather complementary mode.

[lzap]

@natoscott can you give me hint with this one? I very much like the idea of "offline" data collection. I can write a super simple code that will persist all statsd measurements in an (archive) file and then Perl script that will import this into PCP. This could be the default setting for "ad-hoc" monitoring when performance issue appears in our project to let users quickly record "something" we can easily investigate with PCP.

Now, I can see PCP already have some tools to import data (sar2pcp, sheet2pcp), to not reinvent the wheel, I was thinking to store it in some already known (and efficient) format. What would be your recommendation? Thanks

[lzap]

I quickly hacked together a simple statsd seriazer based on MessagePack binary format which I can easily pick up in C or Perl. The PMI API does not seem to be complex, it might be possible to write a Ruby FFI wrapper, not sure how safe is this to do from application context tho.