CUDA VMM Allocation Telemetry

CUDA VMM allocation telemetry

BeeLlama's validation path can report live, mapped, and high-water CUDA VMM allocation from llama-bench. Use it to distinguish smaller logical reservations from pages the driver actually unmapped.

Important

CUDA REQUIRED: These counters describe CUDA VMM behavior. They are not a portable allocator metric and have no meaning for a non-CUDA build.

Behavior

Request it through llama-bench --kv-memory. It never changes allocation or arithmetic. Measure at defined checkpoints because backend pools can retain mappings after logical release.

This is separate from live-context workspace: live-context sizing is a runtime allocation policy, while VMM telemetry is an observation tool used to prove whether physical device mappings followed the logical change.

Implementation

• Benchmark reporting: tools/llama-bench/llama-bench.cpp
• CUDA allocation checkpoints: ggml/src/ggml-cuda/common.cuh
• Validation integration: scripts/feature_validation/telemetry.py

This fork addition is currently published only on beellama/main. Review history: PR 6 and the later decoupling work in PR 19.

For ordinary CUDA build settings, use the upstream build guide.

Feature index