Tags: jjziv/refinery
Tags
Prepare v1.1.0 release (honeycombio#234) Adds changelog for v1.10 and v1.1.0
Make compression of peer forwarding configurable, rather than hard co… …ded (honeycombio#208) We're seeing substantial data transfer between peers in our refinery cluster, far more than we saw going to honeycomb when we were using hosted refinery. We're currently running 3 c5.2xlarge instances of refinery, in separate AZs, and each one seems to be sending ~7MB/s of data to its other 2 peers. Some back of the envelope math suggests this could generate a bill of (7MB/s * 3600 * 24 * 30)/1000MB per GB * $0.02/GB = $362.88 for 1 node to talk to one peer. Given each node has 2 peers, and there are three nodes, it seems plausible that the bandwidth alone could cost in the ballpark of $2,172 - about 3 times the cost of running the cluster, without even considering the cost of transmitting the sampled-in data to Honeycomb. While looking through the source code I noticed that refinery explicitly opts out of using gzip compression when communicating with peers. https://github.com/honeycombio/refinery/blob/3d85a7bb00141f810951eee402ae859aad74e9d5/cmd/refinery/main.go#L150-L153 It seems [this change was made][original-disable] in February 2019. A few months later [libhoney-go was changed to use zstd for compression][libhoney-zstd] by default when sending data to honeycomb's API. The PR that introduces zstd references substantial performance improvements, so I'm assuming Honeycomb had some performance issues with gzip compression, disabled it on several of their services, switched to using zstd, but never got around to enabling it in refinery...? By the looks of it libhoney-go is currently sending zstd compressed data to refinery, and refinery uses it to compress data sent to honeycomb, so I assume we won't run into any substantial performance issues by also using it on peer-to-peer communication. I've introduced a config option to control this in case there are legitimate situations where compression has to be disabled to prevent over-utilization of CPU. fixes honeycombio#206 [original-disable]: honeycombio#21 [libhoney-zstd]: honeycombio/libhoney-go#57
Add a TotalThroughputSampler. (honeycombio#185) * Add a TotalThroughputSampler to try to avoid being rate limited when we hit sudden peaks
Handle all the datatypes in the EMA sampler (honeycombio#186) Now that we support msgpack the types that might be in the data hash are more varied than they used to be. Adds the ability to format any type as a string in order to handle this
Rules Base Sampler (honeycombio#151) Adds new sampler type, which implements some basic rules based sampling.
PreviousNext