LD_PRELOAD doesn't work on Fedora/gcc

[egel-lang]

Dear people,

I am working on an interpreter that now piggybacks on c++ for garbage collection (through std::shared_ptr). I am considering moving over to mimalloc and a concurrent reference counting garbage collector. Unfortunately, initial experiments fail on Fedora/gcc intel.

Why doesn't the following work?

[marco@woof ~]$ time env MIMALLOC_VERBOSE=1 LD_PRELOAD=/usr/lib64/libmimalloc.so.2.0 egel -e "import \"prelude.eg\";; using List;; foldl (+) 0 (from_to 0 1000000)"
mimalloc: option 'show_errors': 0
mimalloc: option 'show_stats': 0
mimalloc: option 'eager_commit': 1
mimalloc: option 'eager_region_commit': 1
mimalloc: option 'reset_decommits': 0
mimalloc: option 'large_os_pages': 0
mimalloc: option 'reserve_huge_os_pages': 0
mimalloc: option 'reserve_huge_os_pages_at': -1
mimalloc: option 'reserve_os_memory': 0
mimalloc: option 'segment_cache': 0
mimalloc: option 'page_reset': 0
mimalloc: option 'abandoned_page_reset': 0
mimalloc: option 'segment_reset': 0
mimalloc: option 'eager_commit_delay': 1
mimalloc: option 'allow_decommit': 1
mimalloc: option 'reset_delay': 25
mimalloc: option 'segment_decommit_delay': 500
mimalloc: option 'use_numa_nodes': 0
mimalloc: option 'limit_os_alloc': 0
mimalloc: option 'os_tag': 100
mimalloc: option 'max_errors': 16
mimalloc: option 'max_warnings': 16
mimalloc: process init: 0x7f70f40a5740
mimalloc: secure level: 0
500000500000
heap stats:    peak      total      freed    current       unit      count   
  reserved:      0          0          0          0                            ok
 committed:      0          0          0          0                            ok
     reset:      0          0          0          0                            ok
   touched:      0          0          0          0                            ok
  segments:      0          0          0          0                            ok
-abandoned:      0          0          0          0                            ok
   -cached:      0          0          0          0                            ok
     pages:      0          0          0          0                            ok
-abandoned:      0          0          0          0                            ok
 -extended:      0    
 -noretire:      0    
     mmaps:      0    
   commits:      0    
   threads:      0          0          0          0                            ok
  searches:     0.0 avg
mimalloc: using 1 numa regions
numa nodes:       1
   elapsed:       2.143 s
   process: user: 2.081 s, system: 0.061 s, faults: 0, rss: 236.7 MiB
mimalloc: process done: 0x7f70f40a5740

real	0m2.150s
user	0m2.082s
sys	0m0.066s
[marco@woof ~]$ 

[daanx]

Good to hear -- mimalloc was originally built for the Koka and Lean languages so it should do well for Egel as well :-) Using LD_PRELOAD should just work on Linux -- no idea what goes on here?

• Can you try the latest version of mimalloc? (just pushed version v1.7.5 and v2.0.5).
• Is this using musl?

Thanks

[egel-lang]

1.7.5 works, I think I use glibc since this is a vanilla Fedora install. I am measuring but due to excessive use of memory I get very widely differing results on glibc (20-30 secs) vs mimalloc (20-22 secs).

Right, thanks Daan, have a nice day

[marco@woof mimalloc-1.7.5]$ time env MIMALLOC_VERBOSE=1 LD_PRELOAD=/usr/local/lib64/libmimalloc.so.1.7 egel -e "import \"prelude.eg\";; using List;; foldl (+) 0 (from_to 0 10000000)"
mimalloc: process init: 0x7f13e69f4740
mimalloc: secure level: 0
mimalloc: using 1 numa regions
mimalloc: option 'show_errors': 0
mimalloc: option 'show_stats': 0
mimalloc: option 'eager_commit': 1
mimalloc: option 'eager_region_commit': 1
mimalloc: option 'reset_decommits': 0
mimalloc: option 'large_os_pages': 0
mimalloc: option 'reserve_huge_os_pages': 0
mimalloc: option 'reserve_huge_os_pages_at': -1
mimalloc: option 'reserve_os_memory': 0
mimalloc: option 'segment_cache': 0
mimalloc: option 'page_reset': 1
mimalloc: option 'abandoned_page_reset': 0
mimalloc: option 'segment_reset': 0
mimalloc: option 'eager_commit_delay': 1
mimalloc: option 'reset_delay': 100
mimalloc: option 'use_numa_nodes': 0
mimalloc: option 'limit_os_alloc': 0
mimalloc: option 'os_tag': 100
mimalloc: option 'max_errors': 16
mimalloc: option 'max_warnings': 16
50000005000000
heap stats:    peak      total      freed    current       unit      count   
  reserved:    2.2 GiB    2.2 GiB      0        2.2 GiB                        not all freed!
 committed:    2.2 GiB    2.2 GiB      0        2.2 GiB                        not all freed!
     reset:    1.0 GiB    1.4 GiB  395.5 MiB    1.0 GiB                        not all freed!
   touched:      0          0        2.6 GiB   -2.6 GiB                        ok
  segments:    537        1.0 Ki     1.0 Ki       2                            not all freed!
-abandoned:      0          0          0          0                            ok
   -cached:      0          0          0          0                            ok
     pages:   33.5 Ki    76.7 Ki    76.7 Ki      25                            not all freed!
-abandoned:      0          0          0          0                            ok
 -extended:      0    
 -noretire:      0    
     mmaps:      0    
   commits:      0    
   threads:      0          0          0          0                            ok
  searches:     0.0 avg
numa nodes:       1
   elapsed:      23.023 s
   process: user: 22.520 s, system: 0.487 s, faults: 0, rss: 2.1 GiB, commit: 2.2 GiB
mimalloc: process done: 0x7f13e69f4740

real	0m23.071s
user	0m22.522s
sys	0m0.533s
[marco@woof mimalloc-1.7.5]$ time egel -e "import \"prelude.eg\";; using List;; foldl (+) 0 (from_to 0 10000000)"
50000005000000

real	0m21.372s
user	0m20.764s
sys	0m0.594s

[daanx]

Great :-). also, if you use mimalloc from a language, and you know that the size of the object that you allocate is less than MI_SMALL_SIZE_MAX (1024 bytes on 64-bit) then calling mi_malloc_small is more efficient.