How to use a preallocated piece of memory as a heap/set of heaps?

[santagada]

Is there any way right now to use a pre-allocated piece of memory for mimalloc? maybe just being able to create a heap with overriden mmap and munmap can be enough.

[daanx]

Nice suggestion -- it should be not too hard to provide some callback hooks for this. Especially since this is already used internally in the os.c file and perhaps better for a coming sbrk/wasm API. Do you have a specific scenario in mind? we can use that to drive the API design.

[sehoffmann]

An example would be the use of memory on a specific numa node or is this already possible to set ?

[vinipsmaker]

we can use that to drive the API design

Not him, but what prevents me from using mimalloc right now is to soft-limit the maximum contiguous memory per mi_heap_t.

I'm developing an actor-like system for lua and each lua VM should have a limit on its maximum memory usage, so I'd like to prealloc the whole block at once (as to not introduce contention for a global memory pool) and use some other project to manage this memory region (this is where I'd like to use mimalloc).

The actor-like part also means that I'll need to access this region from other threads (but only one thread at most will access it at any given time so I don't need any mutex to protect the mi_heap_t).

There are many possible changes to mimalloc that would work for me. The preallocated piece of memory is one option.

[daanx]

Interesting; I am interested to add this kind functionality -- also for embedded systems where we only want allocation from a pre-existing memory range. For this, mimalloc currently has "arenas" (in arena.c where one can put pre-allocated memory regions. As of now, this is only used for huge pre-reserved OS pages.

However, I could add an function entry, like: int mi_add_arena( void* memory, size_t size ) where the size needs to be at least MI_SEGMENT_SIZE. This would add such memory region to be available for regular mimalloc allocation.

You need more though I think; you also want to turn off any other allocation from the OS; i.e. if the arena memory is exhausted, start returning NULL from any allocation function. Is this correct?

[daanx]

Rereading your comment, I guess you would like to have a specific mi_heap_t that can only allocate from a specific arena, but no other places. And other heaps cannot allocate from that arena as well?

[vinipsmaker]

You need more though I think; you also want to turn off any other allocation from the OS; i.e. if the arena memory is exhausted, start returning NULL from any allocation function. Is this correct?

Yes, that's correct!

[vinipsmaker]

Rereading your comment, I guess you would like to have a specific mi_heap_t that can only allocate from a specific arena, but no other places. And other heaps cannot allocate from that arena as well?

I'm not familiar with the mimalloc, so I don't know behaviour details at this point. From your description, that makes sense and it is what I want.

What I want is vocabulary (malloc/realloc/free) to manage a reserved memory region that won't be accessed anywhere else. I may access this region from different threads in different moments, but never from two threads at one single time, so I don't need a mutex. If this memory region cannot fulfill some allocation request, the allocation must not fallback to other means (i.e. it must fail).

A preallocated piece of memory could work, but your description of reserved arenas could also work. I'm happy with any choice you deem superior/closer-to-mimalloc-design-principles.

[santagada]

Our use cases is consoles, so it follows your idea of embeded use. I would like to be able to tell mimalloc to use a memory range, and have all heaps use memory from that range. Both the default heaps, but also have a way to create other set of heaps in another area too.

eg:

2GB (from address x to y) to be used by all default heaps
1GB (from address z to w) to be used by a set of heaps I create.

That and a way to walk a heap are the two features that prevent mimalloc from being used in embeded situations.

[playXE]

Is this issue being worked on? I would like to see dlmalloc's like mspace_morecore api and actually have limits for heap, this helps when you want to do lazy sweep gc since you can do sweep when morecore is invoked :)

[vinipsmaker]

Just stumbled upon this:

A custom memory allocator is used that will cap the amount of memory available to Lua scripts in order to avoid this problem.

— https://lwn.net/Articles/830154/

So there are more Lua VM users stumbling upon this problem as well. However as I progressed on my Lua actor system I found out that the Lua VM API is quite quite quite tricky to work with. Long story short: there are two contexts to run your code — VM context and non-VM context. Code that runs on non-VM context cannot recover from allocation failures and your program would likely to crash. I can provide details if requested, but I don't think I should be flooding mimalloc issue tracker with Lua-related stuff (plus I'm close to release date so even if there is interest I think having public code samples would help to explain the thing).

The solution I came up with was designing this API pattern for code that runs in non-VM context:

allocator->enable_reserved_zone();
// many Lua VM calls such as lua_newtable()
allocator->try_reclaim_reserved_zone();
if (allocator->used_reserved_zone_size() != 0)
    return ENOMEM;
// code to start/enter VM context

The idea is: code running between enable_reserved_zone() and try_reclaim_reserved_zone() is allowed to use extra zones to perform allocation. After try_reclaim_reserved_zone() is called, new allocations happen as usual restricted to the mi_heap_t in hand. There are many ways to implement this. I could add a bit of “allocator logic” myself by wrapping mimalloc functions and falling back to global allocator iff reserved zone is in effect. The upside of this approach is: my old request that you described as "specific mi_heap_t that can only allocate from a specific arena, but no other places" is all that I need and no more.

However given that I'm essentially implementing fallback mechanisms and current mimalloc already implements them I'll probably experiment more here. We'll see.

EDIT:

And there is the alternative to call lua_atpanic() to throw an exception on errors raised from non-VM context of course (if your project is okay to depend on C++).

[biddisco]

I'd like to chip in on this thread as we have a similar use case. For memory buffers that are used for communication - in particular, RMA operations between nodes, we would like to pre-allocate a chunk of memory, register it and assign a heap to use this memory and only this memory - and forbid other heaps from using it. We would want potentially to do this with multiple heaps

• A heap using : Pinned memory for RMA operations
• A heap using : Memory specific to a NUMA node (I am aware that mimalloc is NUMA aware, but we might want finer grained control by explicitly binding certain pages to NUMA nodes)
• A heap using : Memory allocated on another device or unified memory between device/host (GPU/NVDIMM (see memkind))

essentially, to allow mimalloc to act as a more general purpose heap manager for memory where each heap may be bound to a range of addresses. there are some extras, such as registered/pinned memory has a 'key' that is used for remote access, and this would need to be discoverable from the heap.

int mi_add_arena( void* memory, size_t size )

would be great - a well as something like this to make a heap use the arena exclusively

int mi_bind_arena( arena_handle, mi_heap_t, MODE_EXCLUSIVE )

and then we might need something like

mi_get_user_data(arena)

to fetch the registration keys or other mea data about the arena - now we would actually not need that because we'd create a memory block, pass it to mimalloc to use as an arena, bind a heap to it, wrap the metadata and the heap together inside a c++ allocator and store the info we need there - and return the memory registration keys using a fancy pointer if necessary to store address as well as info.

Anyway, if there is any progress on custom area/heap management, it would be very useful to know what plans are in development.

[sighingnow]

It seems that there's already some community effort on this feature: romange@f89ab56.

I'm wondering if such effort could be accepted by the official repo?

Thanks!

[gshanemiller]

I'd like to see this too. For example, I might get some huge page memory for which I'm ultimately responsible to cleanup. But I give it to mimalloc's heap function set and say: here's X bytes. Use it for malloc/free. Then I use that memory via mimalloc. In this way I might better guarantee alignment; I can know the base address and so on. See:

#596

[sighingnow]

FYI: this feature can be done with mi_manage_os_memory.