Expand Sample documentation

[abh]

It'd be nice with a few lines in the readme of how to use the samples. I'm trying to get a "recent data" histogram, but using metrics.NewExpDecaySample(600, 0.015) gives me almost exactly the same data as metrics.NewUniformSample(1800) (I add data to the histogram once a second). I expected them to be less similar, so I'm wondering if I'm doing it wrong.

Specifically then I expected Max/Min to "decay", too.

"since start" is the uniform sample, and "recent" is the ExpDecaySample from above:
http://ord1.ntppool.net:8053/status

Before I looked properly at the code and read the codahale documentation I also setup a version with three different uniformsamples (600, 3600 and 86400 reservoirs): http://zrh2.ntppool.net:8053/status -- this was useful for showing that at least for my use the reservoirsize doesn't seem to matter too much).

Would it make sense for me to implement a sliding window reservoir for my "what's the data been in the last X minutes" use? Since I only update the histogram once a second it's not that much data.

Anyway, a couple of lines in the documentation with recommendations for how and when to use the different sample types would be really helpful.

For reference my code updating the metrics is in https://github.com/abh/geodns/blob/master/metrics.go

[pkieltyka]

Yes, please offer more documentation.. I was just trying this with stathat / librato, and I had to read the code to figure out how to use the lib. Nice work tho.

[Dieterbe]

Would it make sense for me to implement a sliding window reservoir for my "what's the data been in the last X minutes" use? Since I only update the histogram once a second it's not that much data.

yes please. note that coda hale's "Sliding Window Reservoirs" doc says A histogram with a sliding window reservoir produces quantiles which are representative of the past N measurements.
i.e. N measurements, not N seconds or minutes.

I report my metrics every second to graphite, and so i want my timers's min/max/percentiles/etc to be a reflection of the past second. (I would be happy with a solution wether it samples or not).
This is basically the same as @abh's request. I read the coda hale docs but couldn't really find how to do it. the documentation for the go-metrics exp decay window is a 12 page paper, and for the uniform sampling one is a 21 page paper.

[Dieterbe]

@abh:
i read the exponential forward decay paper, and I understand about half of it.
the implementation seems a bit confusing, as far as i understand t0 = landmark L and alpha actually seems to be lambda in the formula. there's some other stuff with the implementation i don't get (like it doesn't seem to do the negation like in the official formula) but anyway in your example
metrics.NewExpDecaySample(600, 0.015) if understand correctly what's going on, 0.015 seems like a very low alpha value, try something like 0.5 or 0.8 instead of 0.015, maybe that's why it retains so much information. also this doesn't really solve the problem of "only process points from within the last X seconds/minutes", because it does exponential decay.

the algorithm R was a bit easier to understand, but also doesn't seem to do what we want. I don't think either algorithm does what we want, although in the forward decay paper there's mention of a window-based decay. That seems to be what we need, but there's no implementation yet for it, that I can see. so maybe we should give that a shot.

another thing I noticed, the R algorithm, once past the initial reservoirsize of samples uses a TRUNC function, and depending on the outcome of that, adds or doesn't add the new value to the reservoir. they don't explain what TRUNC is supposed to do but it looks like they select an array pos between 0 and current-timestamp, which may or may not be a valid index of the reservoir. only if it fits in the reservoir, do they add the value. The go-metrics code on the other hand, always adds the new value, which seems more like what we want, but interesting that there's this difference. they later optimize this by skipping over records (which go-metrics doesn't do).

so I'm thinking of either attempting to implement the windowing variant of the forward decay function,
or perhaps a very simplistic sample container that reset itself after it takes a snapshot (so that at every snapshot, you get the values since the last snapshot). obviously if you have multiple reporters running at the same time this would need more fine tuning.

[mihasya]

Anyone fancy at least a simple PR improving the explanation for how that stuff works? Perhaps a brave soul that has read the paper? @Dieterbe

[mihasya]

(combining with #97 )