historical data

[bf4]

Issue by imajes
Tuesday Jan 17, 2012 at 20:34 GMT
Originally opened as asmuth#43

---

Hey,

i see how awesome fnordmetric is for ongoing, current data, but is there any plan/suggested process for importing historical data?

Thanks!

[bf4]

Comment by bernd
Tuesday Jan 17, 2012 at 21:08 GMT

---

I haven't tested it yet, but it might be possible to add the _time parameter to the event payload. Like this.

echo "{\"_type\": \"unicorn_seen\", \"_time\":1326834487}" | nc localhost 1337

It will be set to the current unix timestamp by default.

[bf4]

Comment by imajes
Tuesday Jan 17, 2012 at 21:10 GMT

---

Yeah, I had wondered about that... and still wondering if anyone has any experience / experiments with this kind of historical data?

[bf4]

Comment by paulasmuth
Thursday Jan 19, 2012 at 12:48 GMT

---

Yes, as bernd pointed out you can either just set the _time field, or use the included importer:

If you usually run your fnordmetric app like this:

ruby my_fnordmetric_app.rb run

you can also invoke the "import" task like this:

ruby my_fnordmetric_app.rb import DUMP_FILE=/path/to/newline_seperated_events.json

If you use "progressive/persistent gauges" (any of your gauges has the :progressive => true option) the events in the dump_file need to be in chronological order (first line should be the oldest and last line should be the most recent).

Btw, there is a third task called "log" which you can use to create these log/dumpfiles:

ruby my_fnordmetric_app.rb log DUMP_FILE=/path/to/events_19.01.12.json

I'll add a readme section about this in the evening.

[bf4]

Comment by imajes
Thursday Jan 19, 2012 at 15:03 GMT

---

This is super helpful, thanks! What would also help is a bit more docs on how best to convert different kinds of data into graphs. :)

On Jan 19, 2012, at 7:48, Paul Asmuthreply@reply.github.com wrote:

Yes, as bernd pointed out you can either just set the _time field, or use the included importer:

If you usually run your fnordmetric app like this:

ruby my_fnordmetric_app.rb run

you can also invoke the "import" task like this:

ruby my_fnordmetric_app.rb import DUMP_FILE=/path/to/newline_seperated_events.json

If you use "progressive/persistent gauges" (any of your gauges has the :progressive => true option) the events in the dump_file need to be in chronological order (first line should be the oldest and last line should be the most recent).

Btw, there is a third task called "log" which you can use to create these log/dumpfiles:

ruby my_fnordmetric_app.rb import DUMP_FILE=/path/to/events_19.01.12.json

I'll add a readme section about this in the evening.

---

Reply to this email directly or view it on GitHub:
asmuth#43 (comment)

[bf4]

Comment by knewter
Thursday Feb 16, 2012 at 14:36 GMT

---

I just posted a gist showing how I'm doing this currently (obv fake data): https://gist.github.com/1845317 /cc @imajes @bernd @paulasmuth

[bf4]

Comment by jonlemmon
Thursday Mar 21, 2013 at 04:01 GMT

---

Is there a reason why this issue has been left open? Looks like it's sorted right?