Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
too much cpu used by hitting elasticsearch for every metric update #96
When I'm sending ~600 metrics every second (just a single host with collectd and 1s update interval) cyanite jvm consumes about 30% of a single core + es jvm consumes another 30%. It looks like every time metric arrives to cyanite it issues a request to es which is not really needed (same set of metrics is being reported every second).
16 Mb is quite underestimated in fact - there will be quite some overhead (arrays, hashtables, pointers, etc). But the idea itself is viable. In fact I became somehow desperate about these performance issues and ended up with implementing my own writer to C* (with ~ same functionality as cyanite) with expiring paths cache. Probably, mostly because I'm not clever enough to master clojure:-)