statsd style reporting

[jpillora]

I know it goes against the intended design of being mostly idle, so close if no - however, would (batched?) transmission of trace data (during low cpu time?) over udp to a remote server ever be in scope? What's your solution for historical data?

[tj]

nah, it's tempting to bunch logging/stats/tracing into once since they're related in most ways but we'll have to try and keep it focused as a live debugging tool

[jpillora]

fair enough

[jpillora]

just curious though, what's your solution for logging historical data?

[tj]

We slap statsd-style counters on most things but if you have say 15 async calls that make up one task, it gets a little crazy to report on all of those, and if you have runtime issues there's often a lot of questions that can't be answered with logging or stats.

https://github.com/component/trace is probably more what you're thinking, I wouldn't mind working on similar for node with statsd support, we use stackdriver right now but proxying through statsd would be fine

[jpillora]

Damn, and do you use statsd as the backend? Statsd is popular but doesn't feel very nodey - feels like there's too many dependencies - weird architectural decisions made. I would love to see a complete statsd-compatible node stack, with client->server->database->real-time graphing/querying.

We're currently writing a round-robin-like database on top of leveldb as a statsd backend, which could be one piece of the puzzle. Unfortunately though, it looks like we're still a ways away from it being a simple npm install statsd-stack!

[tj]

Nah we only use statds for NSQ right now. statsd itself is definitely sloppy IMO, but there's even a go statsd, I haven't really looked at the client code but I'm sure that could be reworked as well