-
Notifications
You must be signed in to change notification settings - Fork 79
Conversation
I think I'm missing something here. I've tried both es-rest and es-native and I never get anything in elasticsearch. I see GET to elasticsearch when using es-rest, which results in a 404 (it's a new stat, so not surprising), but never any PUT/POST to create it. echo "local.random.diceroll 3 (when using es-rest) (when using es-native) |
I added some debugging code to track down where it was missing the PUT for paths not in elasticsearch. When I build with this change, it starts working... O_o diff --git a/src/org/spootnik/cyanite/es_path.clj b/src/org/spootnik/cyanite/es_path.clj
|
@tmonk42 what clojure and what ES are you using? My clojure isn't super strong, but might make sense if something in there is lazy? I will double check, but I am running this in prod and am pretty sure it is working. |
ES 0.90.12 and built with: Leiningen 2.3.4 on Java 1.6.0_65 Java HotSpot(TM) 64-Bit Server VM |
If you have a chance, try with Java 1.7? es-rest should work regardless of On Tue, Jun 17, 2014 at 2:20 PM, Nathan Haneysmith <notifications@github.com
|
Rebuilt with Java 1.7 and pointed at Elasticsearch 1.2.1 and it works (and is ridiculously faster). Thanks! |
Yup will do so shortly
|
Just tested this patch out, I am seeing around an 8x change in performance. I have a little app I wrote that shoves in 20k metrics as fast a possible continuously and will wait for then to be flushed. I generally see delays as high as 80s periodically waiting for data to be processed/flushed. With the new set of patches I am seeing the first delay around 40s then finally around 10s. Still not 100% though, a normal carbon relay can handle a lot more then this so really hoping we can get to the bottom of some of the slow downs. |
Tested that PR with my riemann installation. Works damn good, reduced ES load by 1000% |
Cache paths for elastic search store
thanks addison ! |
As pointed out in #35, the elastic search path store has to hit elasticsearch for each metric.
This was pretty bad, and more testing showed that it was a big bottleneck for writes.
This implements a simple cache so that after writing, only one check is needed.
One thing to note, just realized that currently, writing paths to elasticsearch currently isn't tenant aware. This doesn't fix that, but expect a patch soon :)