Memory

Memory

Strings, objects, arrays, banks and everything the runtime does internally ask for memory very often, and every request to the system allocator costs a call into the operating system's heap. BlitzLLVM answers most of them itself. The sizes are split in three:

|=== | Tier | Size | Where it comes from

| Tiny | up to 2 KiB (less than a page) | size classes of 16 bytes up to 128, then four steps per doubling, cut from 64 KiB spans | Small | up to 32 KiB (less than the 64 KiB allocation granularity) | size classes in four steps per doubling, cut from 256 KiB spans | Normal | more | the system allocator as it is; these are rare enough that nothing is gained |===

Free blocks are kept in a list per thread and size class, so allocating and freeing take no lock and make no system call. A thread that frees much more than it allocates (for example the main thread freeing what a loader thread of AsyncLoading made) hands batches of free blocks to a shared pool that the other threads take from. Spans are never given back to the system, so a program that once needed a lot of small blocks keeps that memory for later small blocks. Memory that comes from the system for the first time is already zero, so calloc of fresh blocks does not clear it again.

All spans lie in one address range that is reserved when the program starts (16 GiB of address space on 64 bit systems, nothing is used until a span is needed; if the system refuses it the program falls back to smaller ranges, and in the end to the system allocator). A pointer is the allocator's when it lies in that range, which is one comparison. So free and realloc of a block that came from somewhere else are passed on to the system and keep working.

What uses it

• new and delete of the runtime (all forms, also aligned and sized) on every platform.
• malloc, calloc, realloc, free, posix_memalign, aligned_alloc and the other C functions on Linux with glibc: they are replaced for the whole program, so the libraries it loads (FFmpeg, the image and audio codecs, the Vulkan driver) use the allocator as well. This can be turned off when building with -DBLITZ_REPLACE_MALLOC=OFF.
• The C code of the runtime itself (also the parts that are compiled with the program) calls the allocator directly on every platform.
• SDL, and with it everything that allocates through SDL_malloc, is pointed at the allocator with SDL_SetMemoryFunctions where the C functions are not replaced.

What cannot be reached where the C functions are not replaced (Windows, macOS): FFmpeg allocates with av_malloc, which has no hook; the codec libraries of SDL_image and SDL_mixer (libpng, libwebp, libjxl, libavif, Opus and others) call the C library themselves, and the allocator hooks they have (png_set_mem_fn, zalloc, JxlMemoryManager) are only reachable by the code that calls them, which is SDL_image; memory the Vulkan driver allocates for itself is the driver's. They keep using the system allocator, which is safe.

Debugging

These environment variables are read when the program starts:

|=== | Variable | Effect

| BLITZLLVM_DEBUG_ALLOC=1 | Freed blocks are filled with a pattern; a block that is written to after it was freed, or freed twice, stops the program with a message when it is reused or freed. New blocks are filled with another pattern, so reading what was never written stands out. The blocks that were not freed are reported when the program ends. Slower. | BLITZLLVM_DEBUG_ALLOC_STATS=1 | Counts the blocks and bytes that were not freed and reports them at the end. | BLITZLLVM_ALLOC_SYSTEM=1 | Everything goes to the system allocator, for tools that watch it (valgrind, AddressSanitizer, heap profilers). |===

The blocks that are reported at the end include the ones the C++ library keeps for the whole run, so a few dozen are normal.