Parallel Mapped MMQ Compilation

Parallel mapped-MMQ compilation

The mapped expert path needs many MMQ template variants. Serial compilation of the translation unit caused one CUDA object to dominate clean build time. This feature splits mapped-MMQ instantiations so the build system can compile them concurrently.

Important

CUDA REQUIRED: This is a CUDA build-time optimization. It does not alter runtime kernel selection or support on other backends.

Behavior

The split preserves the same externally selected MMQ variants and kernel arithmetic. Its purpose is to expose independent compiler work to the existing build parallelism, not to add another runtime dispatcher. Incremental builds still benefit from ccache when compile commands and generated inputs match.

A parallel build can consume substantial host RAM. The project workflow caps local builds at -j20; a smaller value may be necessary on lower-memory systems. This feature does not make one large template instantiation cheap, but it avoids putting all mapped variants behind one serial compilation edge.

Branch implementations

Branch	Implementation
moe-cache	ggml/src/ggml-cuda/mmq.cu, ggml/src/ggml-cuda/mmq.cuh

Implementation commit: d4e438313.

For general CUDA build options, use the upstream build documentation.

Feature index