high mem_fragmentation_ratio

[lw5885799]

version: 5.0.2
problem: high mem_fragmentation_ratio

info memory:
used_memory_human: 24.69M
used_memory_rss_human: 236.4M
mem_fragmentation_ratio: 9.59
mem_allocator: libc

I checked some article. Usually, the mem_fragmentation_ratio is between (1- 1.6). I have no idea why it happenned.
how can I get log or other way to find the cause?

[oranagra]

@lw5885799 We currently have limited visibility into the internals of libc malloc (a lot more details when using jemalloc).
Maybe we can figure something out by looking at /proc/<pid>/smaps

[lw5885799]

@oranagra is there any reference to analyze the proc//smaps. I know few about this memory log. I mean how to find the cause of high mem_fragmentation_ratio by the smaps log.

[oranagra]

just see which mapping consumes the majority of RSS and if you can conclude that's the libc allocator heap or not.

maybe you'll be able to prove that it's related to some other library that got loaded, or maybe that's a memory mapped file.

if that's the allocator then you need to find a way to look into it's internals.
for jemalloc we have je_malloc_stats_print which exposes a lot of data. not sure what is the equivalent for libc malloc, but if you find one, feel free to make a PR for the MEMORY MALLOC-STATS command..

btw, why not use jemalloc?

[lw5885799]

Thanks. I will try this. That guy who building the cluster accidently changed the allocator in product environment. We have changed allocator to jemalloc. Still, want to know what happened

[lw5885799]

I use the memory doctor command in a 4.65 mem_fragmentation_ratio cluster. It says after a memory peak, mem_fragmentation_ratio would be high. Also, it says it is a normal and harmless isuess. As long as I fill it up, mem_fragmentation_ratio goes down. It happened with a jemalloc allocator. I wonder if malloc would lead to the same thing?

My old cluster with malloc had a used_memory_rss 5.5GB used_memory_peak 4.9GB and used_memory 39MB. The mem_fragmentation_ratio reached to 143. Is it still a normal thing?

[oranagra]

if the rss matches the peak memory, it could just be fragmentation, or some issue with the allcator.
but the numbers seems very unlikely. if we allocated 5gb, and then released 90% of that memory, i assume there must be some pages that the allocator can return to the OS.
in other words, it seems unlikely that it'll keep so much memory as rss.

it's hard to tell for sure because we don't have a lot of info with libc allocator.
p.s. when using jemalloc, redis has an activedefrag feature that can solve this.