-
Notifications
You must be signed in to change notification settings - Fork 182
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
consider supporting local-core metrics #102
Comments
Anyone working on this? I would like to contribute as my project depend heavily on this. |
As far as I know there is no-one working on this @jiacai2050. Before you do, would you mind describing your use-case? This crate supports local metrics via the local module which operates on non-thread-synchronized memory, thus bypassing the performance cost of atomic operations, eventually being flushed back to their global thread-synchronized counterparts. I am not aware of a use-case that can not be achieved with the local module, but only with the currently missing idea of metrics local to a given core or thread. |
@jiacai2050 I think you can try with thread local metrics + static metrics. According to our experiments this is already very efficient, although theoretically core-local is even better than thread local :) |
@mxinden @breeswish Thanks for your suggestion. |
@jiacai2050 The current implementation of the thread local metrics provided by this crate is an auto-flush one. Every time you update a thread local metric, the thread local last update time is checked, and then the thread local metric is synced to the global one if specified interval is elapsed. For both public Push and Pull interfaces, the value in the global one is used. This kind of thread local metric avoids frequently updating a global shared memory, i.e. reduces contention to improve performance. It is mostly useful if you update your metrics very frequently in multiple threads (e.g. 1000 times per sec, updated by multiple worker threads). Static metrics macros in this crate is another story. Normally when you use metrics with labels (i.e. metrics vectors), the label needs to be put or retrieve from a HashMap in runtime. However, in many scenarios the metric labels are deterministic and known at compile time. Static metrics macros help you define these static metrics easily, in a way similar to normal metric vectors, without the need to write many boilerplates. It saves HashMap lookup cost. The two techniques (static metrics and thread-local metrics) can be used together to achieve the best performance this crate offers. |
Now using metrics across multi threads has a low performance, so we have to use local metric and then flush to the global metric sometimes, it is not convenient, maybe we can use core-local metrics, But I don't know whether it is easy to do or not.
See http://rocksdb.org/blog/2017/05/14/core-local-stats.html
The text was updated successfully, but these errors were encountered: