My issues and what I had to do to get Qwen3.8-27B /Qwen3.8-27B-MTP-ONLY-Q6_K.gguf /Qwen3.8-27B-MTP-ONLY-Q4_K_M.gguf to work with Llama.cpp

[infinitelayerworks]

llama.cpp / WSL CUDA: Qwen3.8-27B garbage output — DeltaNet CUDA bug, not MTP-specific

I ran into a pretty confusing issue with Qwen3.8-27B in llama.cpp under WSL + CUDA on an RTX 3090, and I wanted to document the fix because it can easily look like an MTP, quantization, driver, or corrupted-model problem.

Short version

Qwen3.8-27B would:

• Load normally
• Use the expected amount of VRAM
• Run at full GPU inference speed
• Show no crash or obvious CUDA error

…but every completion was corrupted garbage.

Examples from the logs:

/Q i
and:
ance iurnesNSE){''),('-ract

The problem turned out not to be MTP.
The base Qwen3.8-27B model was already broken on CUDA before MTP speculative decoding was involved.
The actual issue was an older llama.cpp CUDA implementation affecting Qwen3.8's DeltaNet/Gated DeltaNet layers.

---

Environment

• OS: WSL
• GPU: RTX 3090 24 GB
• Backend: llama.cpp CUDA
• Old llama.cpp commit: 221f0f6
• Working updated build: around b10450 / commit ece963f41
• Flash Attention enabled in the final working configuration
• Vision/mmproj enabled in the final service
• MTP speculative decoding enabled in the final service

The MTP draft models I was using were:

Qwen3.8-27B-MTP-ONLY-Q6_K.gguf
Qwen3.8-27B-MTP-ONLY-Q4_K_M.gguf

My main Qwen3.8 model testing included Q4_K_XL and Q6_K quantizations.

---

Symptoms

At first everything looked normal.
The model loaded correctly and VRAM usage looked reasonable at roughly 19 GB.
Inference also ran at normal GPU speed.

There was:

• No OOM
• No CUDA crash
• No obvious model-load failure
• No useful warning explaining the corrupted output

But the generated tokens were garbage.

That made it very easy to suspect:

• MTP/speculative decoding
• A bad GGUF
• The particular quantization
• NVIDIA drivers
• WSL
• Flash Attention
• mmproj/vision
• Chat template issues

Those turned out not to be the underlying problem.

---

Important finding: MTP was NOT the cause

I made a stripped-down dedicated test with:

• No MTP
• No mmproj
• Minimal flags
• Plain model inference

The results were still broken.

From llama_test.log:

Q4_K_XL without MTP

unparsed Content-only output: /Q i

Garbage.

Q6_K without MTP

Also failed and produced corrupted output.

So both tested base-model quantizations were already broken with speculative decoding completely removed.

Removing:

--spec-type draft-mtp

did not fix the model.

This was the key clue that the apparent MTP problem was actually happening deeper in the normal model inference path.

---

Main service WITH MTP showed the same corruption

My normal service used Q4_K_XL + MTP.

From llama11501.log:

unparsed Content-only output: ance iurnesNSE){''),('-ract
and peg/native reasoning output degraded to things like:

followed by corrupted generation.

So I had:

Q4_K_XL + MTP → garbage

but also:

Q4_K_XL without MTP → garbage

and:

Q6_K without MTP → garbage

That effectively ruled out MTP as the primary failure.

---

What was actually broken

Qwen3.8-27B uses a hybrid architecture containing DeltaNet / Gated DeltaNet-style layers in addition to normal attention.

The llama.cpp revision I was running:

221f0f6

had a problem in the CUDA execution path used by those DeltaNet layers.
Once those layers produced incorrect results, the rest of the model generation was corrupted.
That's why the symptoms were so strange:

The model could load successfully and run quickly while still generating completely invalid output.
Models that didn't exercise the same DeltaNet CUDA path could still work normally on the exact same system.

---

How I confirmed the GGUF files themselves were good

I tested the same GGUF model using the Vulkan/non-CUDA backend from the b10448 release build.
The model immediately produced completely normal text.
That was the strongest diagnostic test.

Same:

• GGUF
• Model weights
• Quantization
• Machine

Different backend.

CUDA

garbage output

Vulkan/non-CUDA path

correct output
`

That strongly pointed away from the model files themselves and toward the CUDA implementation.

---

Another problem: my llama.cpp repository was a shallow clone

This made the issue much harder to diagnose.
The llama.cpp repo had originally been cloned using:

--depth 1

So it only had a minimal amount of Git history locally.
I kept running:

git pull

and Git kept responding:
Already up to date

That made it look like llama.cpp was current.
It wasn't.
The checkout was actually hundreds of commits behind current master.
One major clue was that the llama.cpp executable identified itself as something like:

build 1

instead of the normal llama.cpp build numbering.

---

Fixing the Git checkout

I fixed the shallow repository with:

cd /opt/llama.cpp

git fetch --unshallow

If there are local modifications:

git stash
`

Then:
git pull origin master
This moved me from:

221f0f6

to:

ece963f

which was approximately:

build 10450

at the time I tested it.

---

Rebuild llama.cpp with CUDA

I rebuilt it with:

cmake -B build
-DGGML_CUDA=ON
-DCMAKE_CUDA_ARCHITECTURES=86
-DCMAKE_BUILD_TYPE=Release

Then:

`
cmake --build build --config Release -j4

For an RTX 3090, compute capability 86 is appropriate for this build.

---

CRITICAL gotcha: replacing llama-server alone did NOT fix it

This part cost me additional troubleshooting time.
After compiling the updated llama.cpp, I replaced the llama-server executable.
The model still produced exactly the same garbage.
The reason was that the new executable was still loading an old libggml-cuda.so through my existing library path.
So in reality I had something like:

NEW llama-server
+
OLD libggml-cuda.so

The CUDA kernels actually doing the model work were therefore still from the broken build.

The fix was to update the matching shared libraries too.

Make sure the new build's libraries are being loaded, including the applicable:

libggml.so
libggml-base.so
libggml-cpu.so
libggml-cuda.so
libllama.so

rather than old copies elsewhere on the system.
On Linux/WSL, this is worth checking with:
ldd /path/to/llama-server

and:

echo $LD_LIBRARY_PATH

If necessary, also search for duplicate copies of the libraries.
This was critical in my case because LD_LIBRARY_PATH was still pointing llama.cpp toward the older build.

---

Result after updating BOTH the binary and shared libraries

Everything started working normally.

Final result on my RTX 3090:

• Clean Qwen3.8-27B output
• ~42.9 tok/s
• CUDA working
• Flash Attention working
• Vision/mmproj working
• MTP speculative decoding working

The MTP-only draft models I am using are:

Qwen3.8-27B-MTP-ONLY-Q6_K.gguf

and:

Qwen3.8-27B-MTP-ONLY-Q4_K_M.gguf

---

Why this was misleading

Initially this looked very much like an MTP compatibility/quantization issue.
But the stripped-down test proved otherwise.
The failure matrix was essentially:

Configuration	Result
Q4_K_XL + MTP	❌ Garbage
Q4_K_XL without MTP	❌ Garbage
Q6_K without MTP	❌ Garbage
Same GGUF through Vulkan/non-CUDA path	✅ Correct
Updated llama.cpp CUDA + matching new .so libraries	✅ Correct
Updated CUDA + MTP	✅ Correct

So the important conclusion is:

If Qwen3.8-27B produces garbage with MTP enabled, test it without MTP before assuming speculative decoding is the problem.

In my case the base model was already broken in the older CUDA backend.

---

TL;DR

If Qwen3.8-27B loads normally in llama.cpp CUDA but every response is garbage:

1. Try the model without MTP.
2. Try another backend if possible.
3. Check your actual llama.cpp commit/build.
4. If your repo was cloned with --depth 1, don't blindly trust git pull saying Already up to date.
5. Run:
   git fetch --unshallow
   git pull origin master
6. Rebuild llama.cpp with CUDA.
7. Update the shared libraries along with llama-server.
8. Check ldd and LD_LIBRARY_PATH to make sure an older libggml-cuda.so isn't still being loaded.

In my case the problem was not the GGUF, not Q4 vs Q6, and not MTP.

It was the older Qwen3.8 DeltaNet CUDA path in llama.cpp, followed by the extra trap of the rebuilt server still loading an old CUDA shared library.