regression? v5.1 doesn't seem to release memory back to the OS? #1398
Labels
Comments
|
Hi, This looks to be caused by application idling combined with the time based decay introduced in 5.0. With the new decay design, jemalloc will return memory back to OS gradually over time. We observed efficiency wins with this feature in most applications. However one edge case is, if the application goes completely idle, there won’t be progress. The issue is amplified with time based decay, since when memory was just deallocated the decay is set to be slow (expecting reuse in near future). The background thread feature should solve this, try adding ‘background_thread:true’ in the malloc conf. See more details in https://github.com/jemalloc/jemalloc/blob/dev/TUNING.md |
|
@antirez FYI, FLUSHALL doesn't immediately release memory to the OS. |
|
Meet the same issue for fixed thread number application (a MySQL proxy), not just for idle application. |
|
The time based decay is triggered by active allocation activities; with the time based decay, when threads (or the underlying arena) go idle-ish, the dirty memory may queue longer than expected. As mentioned the |
|
As a side note, although in our experience time based decay performs better in most cases, we did discuss to combine time based with the previous ratio based decay, to better handle cases like when application goes idle. The time based decay does require regular allocation activities to do its best; the |
thenickdude
pushed a commit
to thenickdude/pve-qemu
that referenced
this issue
Mar 22, 2021
jemalloc does not play nice with our Rust library (proxmox-backup-qemu), specifically it never releases memory allocated from Rust to the OS. This leads to a problem with larger caches (e.g. for the PBS block driver). It appears to be related to this GitHub issue: jemalloc/jemalloc#1398 The background_thread solution seems weirdly hacky, so let's disable jemalloc entirely for now. Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
|
I have encountered the same problem while using jemalloc, yet ptmalloc not. // OS: euler os which based on centos
// COMPILE: g++ a.cpp -g -Wall -std=c++11 -I./jemalloc/include/ -ljemalloc -L./jemalloc/lib/
// jemalloc version: 5.2.1
// TEST:
// $ export MALLOC_CONF="prof:true,prof_active:true,prof_prefix:jeprof.out,stats_print:true,muzzy_decay_ms:3000,dirty_decay_ms:3000"
// $ cat > test.sh << EOF
// > a.out &
// > top -p $! | grep $USER
// EOF
// $ sh test.sh
/**
* this program can allocate 62 gigabytes virtual memory at most, and after it reached its peak, it never falls, even after I had
* released all the allocated memory, the amount of virtual memory keeps not changed.
*/
#include <iostream>
#include <memory>
#include <map>
#include <unistd.h>
using namespace std;
constexpr int SIZE = 1024 * 50;
int i = 0;
int main()
{
{
std::map<int, std::unique_ptr<char[]>> map1;
int i = 0;
cout << "before allocate" << endl;
while (i < SIZE) {
map1.emplace_hint(map1.end(), i, new char[rand() % (2 * 1024 * 1024) + 1]());
++i;
}
cout << "All allocated" << endl;
while (i >= 0) {
map1.erase(i);
--i;
}
cout << "All freed" << endl;
while (i < SIZE) {
map1.emplace_hint(map1.end(), i, new char[rand() % (2 * 1024 * 1024) + 1]());
++i;
}
cout << "All reallocated" << endl;
}
cout << "map deallocated" << endl;
sleep(30);
return 0;
}
before allocate
50797 ddwolf 20 0 271712 212508 2004 R 100.0 0.1 0:00.15 a.out
50797 ddwolf 20 0 4871520 4.3g 2008 R 100.0 1.1 0:03.15 a.out
50797 ddwolf 20 0 9727328 8.5g 2008 R 99.7 2.2 0:06.15 a.out
50797 ddwolf 20 0 13.0g 12.6g 2008 R 100.3 3.3 0:09.16 a.out
50797 ddwolf 20 0 18.5g 16.7g 2012 R 99.7 4.4 0:12.16 a.out
50797 ddwolf 20 0 22.0g 20.8g 2012 R 100.3 5.5 0:15.17 a.out
50797 ddwolf 20 0 26.0g 24.9g 2012 R 100.0 6.6 0:18.17 a.out
50797 ddwolf 20 0 31.1g 29.0g 2012 R 99.7 7.7 0:21.17 a.out
50797 ddwolf 20 0 37.1g 33.2g 2012 R 100.0 8.8 0:24.17 a.out
50797 ddwolf 20 0 44.1g 37.3g 2012 R 99.7 9.9 0:27.17 a.out
50797 ddwolf 20 0 44.1g 41.4g 2012 R 100.3 11.0 0:30.18 a.out
50797 ddwolf 20 0 52.1g 45.5g 2012 R 100.0 12.1 0:33.18 a.out
50797 ddwolf 20 0 52.1g 49.6g 2012 R 100.0 13.1 0:36.19 a.out
50797 ddwolf 20 0 62.1g 53.7g 2012 R 100.0 14.2 0:39.19 a.out
All allocated
50797 ddwolf 20 0 62.1g 55.4g 2020 R 100.0 14.7 0:42.19 a.out
50797 ddwolf 20 0 62.1g 41.7g 2048 R 101.0 11.0 0:45.23 a.out
50797 ddwolf 20 0 62.1g 29.1g 2048 R 101.3 7.7 0:48.27 a.out
50797 ddwolf 20 0 62.1g 17.3g 2048 R 101.0 4.6 0:51.31 a.out
50797 ddwolf 20 0 62.1g 4.9g 2052 R 101.3 1.3 0:54.35 a.out
All freed
50797 ddwolf 20 0 62.1g 2.7g 2060 R 101.0 0.7 0:57.38 a.out
50797 ddwolf 20 0 62.1g 6.5g 2060 R 99.7 1.7 1:00.38 a.out
50797 ddwolf 20 0 62.1g 10.3g 2060 R 100.3 2.7 1:03.39 a.out
50797 ddwolf 20 0 62.1g 14.1g 2060 R 100.0 3.7 1:06.39 a.out
50797 ddwolf 20 0 62.1g 17.9g 2060 R 99.7 4.7 1:09.39 a.out
50797 ddwolf 20 0 62.1g 21.7g 2060 R 100.3 5.8 1:12.40 a.out
50797 ddwolf 20 0 62.1g 25.5g 2060 R 99.7 6.8 1:15.40 a.out
50797 ddwolf 20 0 62.1g 29.3g 2060 R 100.3 7.8 1:18.41 a.out
50797 ddwolf 20 0 62.1g 33.1g 2060 R 100.0 8.8 1:21.41 a.out
50797 ddwolf 20 0 62.1g 36.8g 2060 R 99.7 9.8 1:24.41 a.out
50797 ddwolf 20 0 62.1g 40.6g 2060 R 100.3 10.8 1:27.42 a.out
50797 ddwolf 20 0 62.1g 44.4g 2060 R 100.0 11.8 1:30.43 a.out
50797 ddwolf 20 0 62.1g 48.2g 2060 R 99.7 12.8 1:33.42 a.out
50797 ddwolf 20 0 62.1g 52.0g 2060 R 100.3 13.8 1:36.43 a.out
50797 ddwolf 20 0 62.1g 55.8g 2060 R 99.7 14.8 1:39.43 a.out
All reallocated
50797 ddwolf 20 0 62.1g 48.3g 2060 R 100.3 12.8 1:42.44 a.out
50797 ddwolf 20 0 62.1g 37.4g 2060 R 101.0 9.9 1:45.48 a.out
50797 ddwolf 20 0 62.1g 27.0g 2060 R 101.3 7.2 1:48.52 a.out
50797 ddwolf 20 0 62.1g 16.2g 2060 R 101.7 4.3 1:51.57 a.out
50797 ddwolf 20 0 62.1g 4.9g 2060 R 102.0 1.3 1:54.64 a.out
map deallocated
50797 ddwolf 20 0 62.1g 222804 2060 S 45.0 0.1 1:55.99 a.out
50797 ddwolf 20 0 62.1g 120968 2060 S 0.3 0.0 1:56.00 a.out
50797 ddwolf 20 0 62.1g 97232 2060 S 0.0 0.0 1:56.00 a.out
___ Begin jemalloc statistics ___
Version: "0.0.0-0-g0000000000000000000000000000000000000000"
Build-time option settings
config.cache_oblivious: true
config.debug: true
config.fill: true
config.lazy_lock: false
config.malloc_conf: "background_thread:true,dirty_decay_ms:0,muzzy_decay_ms:0,lg_extent_max_active_fit:2"
config.opt_safety_checks: true
config.prof: true
config.prof_libgcc: true
config.prof_libunwind: false
config.stats: true
config.utrace: false
config.xmalloc: false
Run-time option settings
opt.abort: true
opt.abort_conf: true
opt.confirm_conf: false
opt.retain: true
opt.dss: "secondary"
opt.narenas: 288
opt.percpu_arena: "disabled"
opt.oversize_threshold: 8388608
opt.metadata_thp: "disabled"
opt.background_thread: true (background_thread: true)
opt.dirty_decay_ms: 3000 (arenas.dirty_decay_ms: 3000)
opt.muzzy_decay_ms: 3000 (arenas.muzzy_decay_ms: 3000)
opt.lg_extent_max_active_fit: 2
opt.junk: "true"
opt.zero: false
opt.tcache: true
opt.lg_tcache_max: 15
opt.thp: "default"
opt.prof: true
opt.prof_prefix: "jeprof.out"
opt.prof_active: true (prof.active: true)
opt.prof_thread_active_init: true (prof.thread_active_init: true)
opt.lg_prof_sample: 19 (prof.lg_sample: 19)
opt.prof_accum: false
opt.lg_prof_interval: -1
opt.prof_gdump: false
opt.prof_final: false
opt.prof_leak: false
opt.stats_print: true
opt.stats_print_opts: ""
Profiling settings
prof.thread_active_init: true
prof.active: true
prof.gdump: false
prof.interval: 0
prof.lg_sample: 19
Arenas: 289
Quantum size: 16
Page size: 65536
Maximum thread-cached size class: 229376
Number of bin size classes: 52
Number of thread-cache bin size classes: 52
Number of large size classes: 180
Allocated: 21047304, active: 27525120, metadata: 75258344 (n_thp 0), resident: 102760448, mapped: 103022592, retained: 66532933632
Background threads: 1, num_runs: 72, run_interval: 1672638180 ns
n_lock_ops (#/sec) n_waiting (#/sec) n_spin_acq (#/sec) n_owner_switch (#/sec) total_wait_ns (#/sec) max_wait_ns max_n_thds
background_thread 5 0 0 0 0 0 1 0 0 0 0 0
ctl 2 0 0 0 0 0 1 0 0 0 0 0
prof 14 0 0 0 0 0 1 0 0 0 0 0
arenas[0]:
assigned threads: 1
uptime: 145584510420
dss allocation precedence: "secondary"
decaying: time npages sweeps madvises purged
dirty: 3000 0 57 14268 1871191
muzzy: 3000 0 0 0 0
allocated nmalloc (#/sec) ndalloc (#/sec) nrequests (#/sec) nfill (#/sec) nflush (#/sec)
small: 21047304 112946 778 112539 776 111066 765 11700 80 2193 15
large: 0 93754 646 93754 646 93754 646 93754 646 0 0
total: 21047304 206700 1425 206293 1422 204820 1412 105454 727 2193 15
active: 27525120
mapped: 103022592
retained: 66532933632
base: 75191520
internal: 66824
metadata_thp: 0
tcache_bytes: 20897888
resident: 102760448
abandoned_vm: 0
extent_avail: 48830
n_lock_ops (#/sec) n_waiting (#/sec) n_spin_acq (#/sec) n_owner_switch (#/sec) total_wait_ns (#/sec) max_wait_ns max_n_thds
large 2 0 0 0 0 0 1 0 0 0 0 0
extent_avail 195091 1345 0 0 3 0 2281 15 0 0 0 0
extents_dirty 280603 1935 842 5 551 3 1679 11 38000655 262073 1000018 1
extents_muzzy 97508 672 0 0 0 0 3 0 0 0 0 0
extents_retained 223175 1539 141 0 56 0 411 2 1000017 6896 1000017 1
decay_dirty 823 5 0 0 0 0 138 0 0 0 0 0
decay_muzzy 756 5 0 0 0 0 128 0 0 0 0 0
base 49346 340 0 0 0 0 3 0 0 0 0 0
tcache_list 4 0 0 0 0 0 1 0 0 0 0 0
bins: size ind allocated nmalloc (#/sec) ndalloc (#/sec) nrequests (#/sec) nshards curregs curslabs nonfull_slabs regs pgs util nfills (#/sec) nflushes (#/sec) nslabs nreslabs (#/sec) n_lock_ops (#/sec) n_waiting (#/sec) n_spin_acq (#/sec) n_owner_switch (#/sec) total_wait_ns (#/sec) max_wait_ns max_n_thds
8 0 8 1 0 0 0 1 0 1 1 1 0 8192 1 0.000 0 0 0 0 1 0 0 5 0 0 0 0 0 1 0 0 0 0 0
16 1 0 0 0 0 0 0 0 1 0 0 0 4096 1 1 0 0 0 0 0 0 0 3 0 0 0 0 0 1 0 0 0 0 0
---
32 2 32 100 0 99 0 2 0 1 1 1 0 2048 1 0.000 1 0 5 0 1 0 0 10 0 0 0 0 0 1 0 0 0 0 0
48 3 4896 102303 705 102201 704 102394 706 1 102 1 1 4096 3 0.024 10221 70 1026 7 25 1 0 11299 77 0 0 0 0 1 0 0 0 0 0
64 4 0 0 0 0 0 0 0 1 0 0 0 1024 1 1 0 0 0 0 0 0 0 3 0 0 0 0 0 1 0 0 0 0 0
80 5 0 0 0 0 0 0 0 1 0 0 0 4096 5 1 0 0 0 0 0 0 0 3 0 0 0 0 0 1 0 0 0 0 0
---
96 6 0 100 0 100 0 1 0 1 0 0 0 2048 3 1 1 0 4 0 1 0 0 10 0 0 0 0 0 1 0 0 0 0 0
112 7 0 0 0 0 0 0 0 1 0 0 0 4096 7 1 0 0 0 0 0 0 0 3 0 0 0 0 0 1 0 0 0 0 0
---
128 8 0 4 0 4 0 4 0 1 0 0 0 512 1 1 0 0 0 0 2 0 0 15 0 0 0 0 0 1 0 0 0 0 0
160 9 0 106 0 106 0 3 0 1 0 0 0 2048 5 1 2 0 7 0 2 0 0 16 0 0 0 0 0 1 0 0 0 0 0
192 10 0 104 0 104 0 6 0 1 0 0 0 1024 3 1 1 0 5 0 2 0 0 21 0 0 0 0 0 1 0 0 0 0 0
224 11 0 0 0 0 0 0 0 1 0 0 0 2048 7 1 0 0 0 0 0 0 0 3 0 0 0 0 0 1 0 0 0 0 0
---
256 12 0 106 0 106 0 5 0 1 0 0 0 256 1 1 3 0 10 0 2 0 0 20 0 0 0 0 0 1 0 0 0 0 0
320 13 960 106 0 103 0 4 0 1 3 1 0 1024 5 0.002 2 0 6 0 1 0 0 12 0 0 0 0 0 1 0 0 0 0 0
384 14 0 0 0 0 0 0 0 1 0 0 0 512 3 1 0 0 0 0 0 0 0 3 0 0 0 0 0 1 0 0 0 0 0
448 15 0 0 0 0 0 0 0 1 0 0 0 1024 7 1 0 0 0 0 0 0 0 3 0 0 0 0 0 1 0 0 0 0 0
---
512 16 0 106 0 106 0 4 0 1 0 0 0 128 1 1 3 0 9 0 2 0 0 19 0 0 0 0 0 1 0 0 0 0 0
640 17 1280 113 0 111 0 7 0 1 2 1 0 512 5 0.003 4 0 10 0 1 0 0 19 0 0 0 0 0 1 0 0 0 0 0
768 18 0 114 0 114 0 5 0 1 0 0 0 256 3 1 4 0 9 0 2 0 0 20 0 0 0 0 0 1 0 0 0 0 0
896 19 1792 110 0 108 0 9 0 1 2 1 0 512 7 0.003 4 0 11 0 2 0 0 21 0 0 0 0 0 1 0 0 0 0 0
1024 20 1024 80 0 79 0 6 0 1 1 1 0 64 1 0.015 5 0 12 0 2 0 0 23 0 0 0 0 0 1 0 0 0 0 0
1280 21 5120 129 0 125 0 26 0 1 4 1 0 256 5 0.015 6 0 16 0 1 0 0 27 0 0 0 0 0 1 0 0 0 0 0
1536 22 0 112 0 112 0 12 0 1 0 0 0 128 3 1 6 0 14 0 2 0 0 27 0 0 0 0 0 1 0 0 0 0 0
1792 23 5376 129 0 126 0 21 0 1 3 1 0 256 7 0.011 10 0 16 0 1 0 0 30 0 0 0 0 0 1 0 0 0 0 0
2048 24 10240 43 0 38 0 10 0 1 5 1 0 32 1 0.156 4 0 10 0 2 1 0 24 0 0 0 0 0 1 0 0 0 0 0
2560 25 5120 141 0 139 0 37 0 1 2 1 0 128 5 0.015 8 0 20 0 1 0 0 32 0 0 0 0 0 1 0 0 0 0 0
3072 26 6144 91 0 89 0 22 0 1 2 1 0 64 3 0.031 8 0 14 0 1 0 0 26 0 0 0 0 0 1 0 0 0 0 0
3584 27 14336 140 0 136 0 25 0 1 4 1 0 128 7 0.031 8 0 19 0 1 0 0 31 0 0 0 0 0 1 0 0 0 0 0
4096 28 8192 47 0 45 0 28 0 1 2 1 0 16 1 0.125 9 0 19 0 2 1 0 34 0 0 0 0 0 1 0 0 0 0 0
5120 29 35840 130 0 123 0 53 0 1 7 1 0 64 5 0.109 10 0 18 0 1 0 0 32 0 0 0 0 0 1 0 0 0 0 0
6144 30 36864 81 0 75 0 55 0 1 6 1 0 32 3 0.187 9 0 17 0 2 1 0 32 0 0 0 0 0 1 0 0 0 0 0
7168 31 28672 117 0 113 0 50 0 1 4 1 0 64 7 0.062 12 0 18 0 1 0 0 34 0 0 0 0 0 1 0 0 0 0 0
8192 32 73728 70 0 61 0 55 0 1 9 2 1 8 1 0.562 14 0 16 0 10 2 0 51 0 0 0 0 0 1 0 0 0 0 0
10240 33 112640 123 0 112 0 86 0 1 11 1 1 32 5 0.343 19 0 17 0 4 2 0 46 0 0 0 0 0 1 0 0 0 0 0
12288 34 61440 90 0 85 0 72 0 1 5 1 1 16 3 0.312 13 0 16 0 6 1 0 43 0 0 0 0 0 1 0 0 0 0 0
14336 35 129024 143 0 134 0 108 0 1 9 1 1 32 7 0.281 17 0 18 0 5 1 0 47 0 0 0 0 0 1 0 0 0 0 0
16384 36 131072 96 0 88 0 89 0 1 8 2 0 4 1 1 14 0 16 0 24 2 0 79 0 0 0 0 0 1 0 0 0 0 0
20480 37 368640 199 1 181 1 201 1 1 18 2 1 16 5 0.562 32 0 18 0 13 1 0 77 0 0 0 0 0 1 0 0 0 0 0
24576 38 417792 188 1 171 1 183 1 1 17 3 1 8 3 0.708 35 0 22 0 24 0 0 105 0 0 0 0 0 1 0 0 0 0 0
28672 39 458752 181 1 165 1 178 1 1 16 1 0 16 7 1 31 0 16 0 11 1 0 71 0 0 0 0 0 1 0 0 0 0 0
32768 40 163840 188 1 183 1 188 1 1 5 3 1 2 1 0.833 37 0 24 0 94 1 0 249 1 0 0 0 0 1 0 0 0 0 0
40960 41 368640 363 2 354 2 369 2 1 9 2 1 8 5 0.562 84 0 39 0 46 0 0 216 1 0 0 0 0 1 0 0 0 0 0
49152 42 688128 330 2 316 2 332 2 1 14 4 1 4 3 0.875 74 0 37 0 82 1 0 274 1 0 0 0 0 1 0 0 0 0 0
57344 43 344064 350 2 344 2 340 2 1 6 1 1 8 7 0.750 66 0 39 0 45 0 0 198 1 0 0 0 0 1 0 0 0 0 0
65536 44 1114112 363 2 346 2 356 2 1 17 17 0 1 1 1 68 0 39 0 363 0 0 819 5 0 0 0 0 1 0 0 0 0 0
81920 45 1228800 619 4 604 4 631 4 1 15 4 1 4 5 0.937 97 0 64 0 155 1 0 470 3 0 0 0 0 1 0 0 0 0 0
98304 46 1081344 657 4 646 4 668 4 1 11 6 1 2 3 0.916 97 0 69 0 328 1 0 819 5 0 0 0 0 1 0 0 0 0 0
114688 47 2179072 663 4 644 4 670 4 1 19 5 1 4 7 0.950 97 0 68 0 166 1 0 495 3 0 0 0 0 1 0 0 0 0 0
131072 48 2359296 589 4 571 3 600 4 1 18 18 0 1 2 1 91 0 61 0 589 0 0 1315 9 0 0 0 0 1 0 0 0 0 0
163840 49 2621440 1121 7 1105 7 1124 7 1 16 8 0 2 5 1 164 1 115 0 560 1 0 1394 9 0 0 0 0 1 0 0 0 0 0
196608 50 3538944 1042 7 1024 7 1056 7 1 18 18 0 1 3 1 156 1 106 0 1042 0 0 2331 16 0 0 0 0 1 0 0 0 0 0
229376 51 3440640 958 6 943 6 970 6 1 15 8 1 2 7 0.937 153 1 98 0 479 1 0 1204 8 0 0 0 0 1 0 0 0 0 0
large: size ind allocated nmalloc (#/sec) ndalloc (#/sec) nrequests (#/sec) curlextents
262144 52 0 3913 26 3913 26 3913 26 0
327680 53 0 3254 22 3254 22 3254 22 0
393216 54 0 3236 22 3236 22 3236 22 0
458752 55 0 3305 22 3305 22 3305 22 0
524288 56 0 3250 22 3250 22 3250 22 0
655360 57 0 6301 43 6301 43 6301 43 0
786432 58 0 6460 44 6460 44 6460 44 0
917504 59 0 6273 43 6273 43 6273 43 0
1048576 60 0 6410 44 6410 44 6410 44 0
1310720 61 0 12797 88 12797 88 12797 88 0
1572864 62 0 12771 88 12771 88 12771 88 0
1835008 63 0 12665 87 12665 87 12665 87 0
2097152 64 0 13119 90 13119 90 13119 90 0
---
extents: size ind ndirty dirty nmuzzy muzzy nretained retained ntotal total
65536 0 0 0 0 0 2 131072 2 131072
---
786432 9 0 0 0 0 1 720896 1 720896
---
1048576 11 0 0 0 0 1 1048576 1 1048576
---
1835008 14 0 0 0 0 1 1835008 1 1835008
---
2621440 16 0 0 0 0 2 5242880 2 5242880
3145728 17 0 0 0 0 1 2949120 1 2949120
---
5242880 20 0 0 0 0 4 18677760 4 18677760
6291456 21 0 0 0 0 1 6225920 1 6225920
7340032 22 0 0 0 0 3 21430272 3 21430272
8388608 23 0 0 0 0 2 15794176 2 15794176
10485760 24 0 0 0 0 2 18874368 2 18874368
12582912 25 0 0 0 0 3 35848192 3 35848192
14680064 26 0 0 0 0 1 14614528 1 14614528
16777216 27 0 0 0 0 1 15532032 1 15532032
20971520 28 0 0 0 0 1 17694720 1 17694720
25165824 29 0 0 0 0 3 72220672 3 72220672
---
33554432 31 0 0 0 0 1 32178176 1 32178176
41943040 32 0 0 0 0 2 73728000 2 73728000
50331648 33 0 0 0 0 4 185466880 4 185466880
58720256 34 0 0 0 0 3 165281792 3 165281792
67108864 35 0 0 0 0 2 120389632 2 120389632
83886080 36 0 0 0 0 1 69795840 1 69795840
100663296 37 0 0 0 0 1 92340224 1 92340224
117440512 38 0 0 0 0 1 105119744 1 105119744
134217728 39 0 0 0 0 1 120389632 1 120389632
---
201326592 41 0 0 0 0 2 370343936 2 370343936
234881024 42 0 0 0 0 2 431489024 2 431489024
268435456 43 0 0 0 0 2 504889344 2 504889344
335544320 44 0 0 0 0 3 894173184 3 894173184
402653184 45 0 0 0 0 3 1101463552 3 1101463552
---
536870912 47 0 0 0 0 2 957087744 2 957087744
---
1073741824 51 0 0 0 0 1 1069416448 1 1069416448
1342177280 52 0 0 0 0 1 1232273408 1 1232273408
---
4294967296 59 0 0 0 0 1 3997433856 1 3997433856
---
8589934592 63 0 0 0 0 2 16642998272 2 16642998272
10737418240 64 0 0 0 0 1 10737418240 1 10737418240
---
15032385536 66 0 0 0 0 2 27380416512 2 27380416512
---
--- End jemalloc statistics --- |
|
Turns out it is because the configure CFLAGS=-fPIC CXXFLAGS=-fPIC --with-lg-page=16 --with-lg-hugepage=21 --with-malloc-conf="background_thread:true,dirty_decay_ms:0,muzzy_decay_ms:0,lg_extent_max_active_fit:2" --enable-prof --enable-debug --prefix=%s/install_release |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi,
I recently upgraded from v4.0.3 to v5.1 (using Redis).
From past experience I remember that when releasing memory, jemalloc was keeping some 12% (1/8) of the current usage for future use, instead of giving everything back to the OS.
But now, i notice that with v5.1 it doesn't seem to return anything back to the OS.
I assume i'm missing something major, but i can't seem to find it in the release notes, or recent commits / issues.
for example, empty process:
Then after doing many allocations (of about 500MB), and then releasing most of them:
In v4.0.3 the
residentused to shrink back to about 14MB.The text was updated successfully, but these errors were encountered: