tests: enable kv_unified for test-backend-sampler

[taronaeo]

While running tests to ensure that my local server is ready to be onboard as a GGML CI runner, I ran into an odd CUDA OOM error for the dist sampling test as shown:

2026-03-15T15:43:18.5103166Z ggml_backend_cuda_buffer_type_alloc_buffer: allocating 5320.00 MiB on device 0: cudaMalloc failed: out of memory
2026-03-15T15:43:18.5103360Z alloc_tensor_range: failed to allocate CUDA0 buffer of size 5578424320
2026-03-15T15:43:18.5103633Z llama_init_from_model: failed to initialize the context: failed to allocate buffer for kv cache
2026-03-15T15:43:18.5103764Z Error running test 'dist': failed to create context

Comparing the dist sampling test across all the other available tests, I noticed that seq_id = 0 has been used everywhere except test_backend_dist_sampling. Setting seq_id from 189 to 0 seem to have solved the OOM error.

cc: @danbev; please let me know if seq_id = 189 was intentional or if there is another approach to solving this :)

Edit: PR has been updated to change only kv_unified = true as suggested #20645 (comment).

[danbev]

please let me know if seq_id = 189 was intentional or if there is another approach to solving this :)

This was just to use something other than 0 which I used in most other test, and I did not take this into consideration.

[ggerganov]

Hm, it should work even with seq_id = 189. Any seq_id up to LLAMA_MAX_SEQ = 256 should work with the unified kv cache. Might be worth to trace down the root cause for this OOM.

[ggerganov]

We are actually not enabling the unified KV cache. This patch should fix it:

diff --git a/tests/test-backend-sampler.cpp b/tests/test-backend-sampler.cpp
index d4cd62c71..58361ae80 100644
--- a/tests/test-backend-sampler.cpp
+++ b/tests/test-backend-sampler.cpp
@@ -89,6 +89,7 @@ struct test_context {
         cparams.n_batch = 512;
         cparams.samplers = configs.data();
         cparams.n_samplers = configs.size();
+        cparams.kv_unified = true;
 
         // If n_seq_max is not specified, calculate it from configs
         if (n_seq_max < 0) {

[taronaeo]

Sorry for the delay. Thank you, that patch fixed it! I'll update this PR to reflect that change :)

[taronaeo]

Merge in a few hours if no further comments :)