Offload KV Cache to GPU Memory no longer works in 0.3.23 Build 3

[WalasPrime]

Which version of LM Studio?
Example: LM Studio 0.3.23 (Build 3)

Which operating system?
Windows 10

What is the bug?
The "Offload KV Cache to GPU Memory" toggle does not seem to be functional after the "Force Model Expert Weights onto CPU" experimental toggle was added.

The use case is CPU inference with the KV cache being stored on the GPU.

Screenshots
Despite "Offload KV Cache to GPU Memory" the lgos show the KV cache being allocated on the CPU side.

Logs

2025-08-14 21:59:09 [DEBUG]
 load_tensors: offloading 0 repeating layers to GPU
load_tensors: offloaded 0/25 layers to GPU
load_tensors:   CPU_Mapped model buffer size = 11536.18 MiB
2025-08-14 21:59:09 [DEBUG]
 llama_context: constructing llama_context
llama_context: n_seq_max     = 1
llama_context: n_ctx         = 131072
llama_context: n_ctx_per_seq = 131072
llama_context: n_batch       = 512
llama_context: n_ubatch      = 512
llama_context: causal_attn   = 1
llama_context: flash_attn    = 1
llama_context: kv_unified    = false
llama_context: freq_base     = 150000.0
llama_context: freq_scale    = 0.03125
llama_context:        CPU  output buffer size =     0.77 MiB
llama_kv_cache_unified_iswa: using full-size SWA cache (ref: https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
llama_kv_cache_unified_iswa: creating non-SWA KV cache, size = 131072 cells
2025-08-14 21:59:09 [DEBUG]
 llama_kv_cache_unified:        CPU KV buffer size =  3072.00 MiB
2025-08-14 21:59:10 [DEBUG]
 llama_kv_cache_unified: size = 3072.00 MiB (131072 cells,  12 layers,  1/1 seqs), K (f16): 1536.00 MiB, V (f16): 1536.00 MiB
llama_kv_cache_unified_iswa: creating     SWA KV cache, size = 131072 cells
2025-08-14 21:59:10 [DEBUG]
 llama_kv_cache_unified:        CPU KV buffer size =  3072.00 MiB
2025-08-14 21:59:10 [DEBUG]
 llama_kv_cache_unified: size = 3072.00 MiB (131072 cells,  12 layers,  1/1 seqs), K (f16): 1536.00 MiB, V (f16): 1536.00 MiB
2025-08-14 21:59:11 [DEBUG]
 llama_context:      CUDA0 compute buffer size =   985.20 MiB
llama_context:  CUDA_Host compute buffer size =   517.65 MiB

To Reproduce
Steps to reproduce the behavior (on a system with CUDA):

1. Load any model with "GPU Offload" set to 0, select "Offload KV Cache to GPU Memory"
2. In Developer Logs notice that the kv cache is being allocated on the CPU side while the expected allocation would be on CUDA

It seems that the KV cache only gets allocated with CUDA proportionally to the level of "GPU Offload", which in this case is 0.

Found workaround
A workaround that works for MoE models only includes utilizing the new "Force Model Expert Weights onto CPU" toggle. This is when you want the model on CPU but the KV cache on the GPU:

1. Set "GPU offload" to max, but also select "Force Model Expert Weights onto CPU"
2. That's it! You should already see the KV cache being allocated proportionally to the number of "GPU offload" level that you've set, while the model remains on the CPU side.

[rllbe]

Set "GPU offload" to max, but also select "Force Model Expert Weights onto CPU"

Works the opposite way for me. With it selected the model has most of its weight offloaded to GTT ram, without it the model fits completely inside vram.

spec: Windows 11 24H2, LM Studio 0.3.24, gpt-oss-20b, RTX 5070 Ti, vram 16GB/GTT 64GB

[MrPoisen]

I experience something similar. Using Non-moe Models, it appears that the KV Cache is always offloaded to the GPU.
GPU Offload set to 0 for Qwen3 4B Instruct:

100k Context, "Offload KV Cache to GPU Memory" set to false

100k Context, "Offload KV Cache to GPU Memory" set to true

50k Context, "Offload KV Cache to GPU Memory" set to false

50k Context, "Offload KV Cache to GPU Memory" set to true

I used Cuda 12 llama.cpp (Windows). After a quick check it appears to happen with the Vulkan engine too.

I'm using LM Studio 0.3.25 (Build 2)