Hi LM Studio team! I wanted to put TurboQuant+ on your radar for a future update. It's a recent implementation of the TurboQuant paper (ICLR 2026) that adds extreme KV cache compression (turbo3 and turbo4) to llama.cpp.
It allows for massive context windows on Apple Silicon and consumer GPUs (e.g., running a 104B model at 128K context on a Mac Studio) with almost zero speed or quality loss by using attention-gated decoding (Sparse V).
The author has a working fork of llama.cpp here: https://github.com/TheTom/turboquant_plus and is preparing an upstream PR.
It would be amazing if LM Studio could support the --cache-type-k turbo3 and --cache-type-v turbo3 flags in the UI once this gets merged into upstream llama.cpp, or perhaps as an experimental engine toggle before then.
Hi LM Studio team! I wanted to put TurboQuant+ on your radar for a future update. It's a recent implementation of the TurboQuant paper (ICLR 2026) that adds extreme KV cache compression (turbo3 and turbo4) to llama.cpp.
It allows for massive context windows on Apple Silicon and consumer GPUs (e.g., running a 104B model at 128K context on a Mac Studio) with almost zero speed or quality loss by using attention-gated decoding (Sparse V).
The author has a working fork of llama.cpp here: https://github.com/TheTom/turboquant_plus and is preparing an upstream PR.
It would be amazing if LM Studio could support the --cache-type-k turbo3 and --cache-type-v turbo3 flags in the UI once this gets merged into upstream llama.cpp, or perhaps as an experimental engine toggle before then.