CPU Optimizations

CPU optimizations

None of this changes the result of a program. Each part has an environment variable that turns it down or off.

CPU features

Programs are compiled for the generic target of their architecture (x86-64 with SSE2, or ARMv8.0 with NEON on arm64). The runtime asks the CPU once which optional instruction sets exist, and code that is faster with one only runs when the CPU has it. Today that is LockBuffer and UnlockBuffer swapping the red and blue channels of 32 bit buffers (SSSE3 and AVX2 on x86-64, NEON on arm64). Every faster version gives the same bytes as the plain one. Floats are never fused into multiply-adds there.

BLITZLLVM_CPU lowers what the runtime may use, so each code path can be tested without owning every CPU. It never adds a feature the CPU lacks. It holds a comma separated list:

Setting	Effect
baseline	No optional features: SSE2 only on x86-64, NEON only on arm64.
v2, v3, v4	x86-64 only: at most the features of the x86-64-v2, -v3 or -v4 level (v4 is AVX-512).
native	What the CPU has. This is the default.
-<feature>	Removes a feature and everything that needs it, such as -avx2, -fma, -sse4.1, -dotprod or -sve2.

For example BLITZLLVM_CPU=v3,-fma runs as an AVX2 machine without FMA. A program started with --profile and BLITZ_FPS=1 prints the result as cpu: x86_64 v3: sse3 ssse3 ... before the first frame report.

Threads

Some work is the same for every element and does not depend on the others. The runtime splits it over a few threads, and the result is the same as one thread makes.

Today that is cutting big surfaces into the runs that the GPU culls (see GPU optimizations). Surfaces with fewer than 4096 vertices or indices are not worth waking the threads for.

BLITZLLVM_THREADS sets how many threads work, the thread that renders included. The default is half of the cores, at most 8, and 1 does everything on the thread that renders.

Loading and saving pictures on other threads is a separate, opt-in feature: Asynchronous loading and Saving pictures.

Memory allocator

Strings, objects, arrays, banks and the runtime itself ask for memory very often. BlitzLLVM answers most requests itself instead of calling the heap of the operating system each time.

• Small blocks (up to 32 KiB) are cut from large spans and kept in free lists per thread. Allocating and freeing take no lock and no system call.
• Larger blocks go to the system allocator.
• Spans that have been entirely free for more than a second are given back to the system. A program that once needed a lot of small blocks (a level that was loaded and freed) does not keep that memory.
• Memory that is given back reads as zero when it is used again.

The allocator is used for new and delete of the runtime on every platform. On Linux with glibc it also replaces malloc, free and the related C functions, so the libraries a program loads use it too. Build with -DBLITZ_REPLACE_MALLOC=OFF to turn that off.

On Windows and macOS the libraries (FFmpeg, the image and audio codecs, the Vulkan driver) keep using the system allocator. That is safe, because free and realloc of a block from elsewhere are passed on.

To debug memory problems, set these before the program starts:

Variable	Effect
BLITZLLVM_DEBUG_ALLOC=1	Fills freed blocks with a pattern. A block written to after it was freed, or freed twice, stops the program with a message. New blocks get another pattern, so reading what was never written stands out. Blocks never freed are reported at the end (a few dozen kept by the C++ library are normal). Slower.
BLITZLLVM_DEBUG_ALLOC_STATS=1	Counts the blocks and bytes that were not freed and reports them at the end.
BLITZLLVM_ALLOC_RELEASE=0	Keeps spans instead of giving empty ones back, to compare memory use over time.
BLITZLLVM_DEBUG_ALLOC_RELEASE=1	Prints a line to stderr for every sweep that gave spans back.
BLITZLLVM_ALLOC_SYSTEM=1	Everything goes to the system allocator, for valgrind, AddressSanitizer and heap profilers.