Memory

Memory

Strings, objects, arrays, banks and everything the runtime does internally ask for memory very often, and every request to the system allocator costs a call into the operating system's heap. BlitzLLVM answers most of them itself. The sizes are split in three:

|=== | Tier | Size | Where it comes from

| Tiny | up to 2 KiB (less than a page) | size classes of 16 bytes up to 128, then four steps per doubling, cut from 64 KiB spans | Small | up to 32 KiB (less than the 64 KiB allocation granularity) | size classes in four steps per doubling, cut from 256 KiB spans | Normal | more | the system allocator as it is; these are rare enough that nothing is gained |===

Free blocks are kept in a list per thread and size class, so allocating and freeing take no lock and make no system call. A thread that frees much more than it allocates (for example the main thread freeing what a loader thread of AsyncLoading made) hands batches of free blocks to a shared pool that the other threads take from. Spans in which every block has been free for more than a second are given back to the system, so a program that once needed a lot of small blocks (a level that was loaded and freed) does not keep that memory for the rest of its run. The address range of a span stays reserved and the span is used again before a new one is taken; its memory reads as zero again, like the memory of a span that was never used. One empty span per size class always stays, so a program that frees and allocates in turns does not give a span back and take it again each time.

The check is not done by malloc and free (they never read the clock): Flip and the places where a program waits (Delay, WaitKey and the like) look at most every quarter second. A span counts as empty when all its blocks are in the shared pool or in the free list of the thread that looks; a block in the free list of another thread keeps its span, which is only a delay since those lists are handed to the pool when they grow or the thread ends. Memory that comes from the system for the first time is already zero, so calloc of fresh blocks does not clear it again.

Delete of an object sets the variable (or field, or array element) it was given to Null, so Delete t followed by t\x stops with Object does not exist. instead of reading freed memory. Variables, fields and array elements count the objects they hold: a deleted object gives its memory back as soon as the last of them lets go, which is at once when only the variable of the Delete held it. Other variables that still point at the deleted object are not changed, using one stops with Object does not exist. in every build. A For Each loop over the type that is running also holds the memory back (it finds its next object through the one it stands on, so Delete t in the loop and deleting other objects of the type work as before; Return, Exit and the end of the loop let go of it), and a program compiled with -d keeps the last 65536 deleted objects of each type as well. A function that returns an object that was deleted gives back Null. Objects that the program never deletes are never freed, as the program or a userlib may still use them.

All spans lie in one address range that is reserved when the program starts (16 GiB of address space on 64 bit systems, nothing is used until a span is needed; if the system refuses it the program falls back to smaller ranges, and in the end to the system allocator). A pointer is the allocator's when it lies in that range, which is one comparison. So free and realloc of a block that came from somewhere else are passed on to the system and keep working.

What uses it

• new and delete of the runtime (all forms, also aligned and sized) on every platform.
• malloc, calloc, realloc, free, posix_memalign, aligned_alloc and the other C functions on Linux with glibc: they are replaced for the whole program, so the libraries it loads (FFmpeg, the image and audio codecs, the Vulkan driver) use the allocator as well. This can be turned off when building with -DBLITZ_REPLACE_MALLOC=OFF.
• The C code of the runtime itself (also the parts that are compiled with the program) calls the allocator directly on every platform.
• SDL, and with it everything that allocates through SDL_malloc, is pointed at the allocator with SDL_SetMemoryFunctions where the C functions are not replaced.

What cannot be reached where the C functions are not replaced (Windows, macOS): FFmpeg allocates with av_malloc, which has no hook; the codec libraries of SDL_image and SDL_mixer (libpng, libwebp, libjxl, libavif, Opus and others) call the C library themselves, and the allocator hooks they have (png_set_mem_fn, zalloc, JxlMemoryManager) are only reachable by the code that calls them, which is SDL_image; memory the Vulkan driver allocates for itself is the driver's. They keep using the system allocator, which is safe.

Debugging

These environment variables are read when the program starts:

|=== | Variable | Effect

| BLITZLLVM_DEBUG_ALLOC=1 | Freed blocks are filled with a pattern; a block that is written to after it was freed, or freed twice, stops the program with a message when it is reused or freed. New blocks are filled with another pattern, so reading what was never written stands out. The blocks that were not freed are reported when the program ends. Slower. | BLITZLLVM_DEBUG_ALLOC_STATS=1 | Counts the blocks and bytes that were not freed and reports them at the end. | BLITZLLVM_ALLOC_RELEASE=0 | Spans are kept, as in versions before the release of empty spans, to compare the memory use over time. | BLITZLLVM_DEBUG_ALLOC_RELEASE=1 | Prints a line to stderr for every sweep that gave spans back: how many, when, and how long the longest was empty. | BLITZLLVM_ALLOC_SYSTEM=1 | Everything goes to the system allocator, for tools that watch it (valgrind, AddressSanitizer, heap profilers). |===

The blocks that are reported at the end include the ones the C++ library keeps for the whole run, so a few dozen are normal.