Add metrics to track memory/heap consumption #22

Open
thinkingfish opened this Issue Jan 14, 2013 · 0 comments

Projects

None yet

1 participant

@thinkingfish
Twitter, Inc. member

Learning about the real memory consumption of Twemcache is important to correctly estimate overhead and avoid paging. To many people's surprise, slab memory doesn't account for the entire heap size in many cases, and it would be helpful to have metrics reflecting actual heap size and its composition.

Aside from slabs, large memcache instances usually allocate a lot of memory into hashtable(s); and for instances with a lot of connections, connection buffer is also a significant source of memory overhead. So it would be nice to have the following metrics for starters:

heap_curr /* total heap size, everything allocated through mc_*alloc */
heap_hashtable /* size of the current hashtable, and if in transition, hashtables */
heap_conn /* connection buffer related overhead */

There are others that could be added, such as slab size (which can currently be computed from slab_curr and slab size), suffix buffer for reply messages, etc. It would be nice to come up a more comprehensive component list, but they probably aren't as important as the above ones.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment