Misc. bug: webui is dropping or omitting model's HTML output

[zzokkolma]

Name and Version

$ ./build/bin/llama-server --version
version: 6684 (606a73f)
built with cc (Debian 14.2.0-19) 14.2.0 for x86_64-linux-gnu

Operating systems

Linux

Which llama.cpp modules do you know to be affected?

llama-server

Command line

./build/bin/llama-server -m /mnt/models/DeepSeek-R1-0528-UD-IQ1_S-00001-of-00004.gguf --ctx-size 20480 --port 8080 --jinja --no-warmup --no-repack
./build/bin/llama-server -m /mnt/models/granite-4.0-h-small-Q6_K.gguf --ctx-size 16384 --port 8080 --jinja
./build/bin/llama-server -m /mnt/models/Qwen3-30B-A3B-Thinking-2507-Q4_K_M.gguf --ctx-size 16384 --port 8080 --jinja

Problem description & steps to reproduce

I use the web UI to attach a ~30KB .txt file with my notes and instructions to create an HTML page with a table showing the results of my notes with some specific columns requested, as well as a summary and analysis.
For thinking models, the web UI correctly shows the reasoning tokens, but frequently drops parts or sometimes even entire final HTML output of the model. I can see that tokens are being produced, because the counters at the bottom of the page are going up, but nothing is showing up.
For non-thinking models some parts of the output might show up (like a single <p>...</p>) or nothing at all.
I also tested sending my query directly to the /v1/chat/completions API endpoint, and that endpoint produces a complete and correct response for the same model and server instance, no tokens missing.
I tested multiple different models - DeepSeek, Qwen, Granite - all of them had some issues, though some of them managed to produce some parts of the output in the web UI sometimes.
Based on the fact that this happens in multiple models on the web UI but the API does not have that problem, I assume there's a problem with handling HTML output in the UI.

First Bad Commit

No response

Relevant log output

[zzokkolma]

Actually, this might be related to #16263 (which I wasn't able to reproduce) as I just had this happen with non-HTML output as well. I also tested this with gpt-oss-120b and that model outputs HTML just fine on the web UI, but it's also faster to output than the ones I tried before. The key might be that I am running CPU inference and loading medium/large models with a few thousand tokens of context up front, which is slow. I'm fine with it being slow, I choose to run it on CPU, but dropping response tokens in web UI is not good.
I'm also using Firefox 140.3.1esr, in case that's relevant.

[ServeurpersoCom]

Actually, this might be related to #16263 (which I wasn't able to reproduce) as I just had this happen with non-HTML output as well. I also tested this with gpt-oss-120b and that model outputs HTML just fine on the web UI, but it's also faster to output than the ones I tried before. The key might be that I am running CPU inference and loading medium/large models with a few thousand tokens of context up front, which is slow. I'm fine with it being slow, I choose to run it on CPU, but dropping response tokens in web UI is not good. I'm also using Firefox 140.3.1esr, in case that's relevant.

We need to make sure to distinguish between a rendering issue (reproduced) and a data loss issue (not reproduced).

The thinking block is already fixed by PR #16483

Yeah, I’ve seen this issue the HTML part gets stripped out of the markdown. Looks like it just needs a sanitize (or similar) step to keep the markdown rendering intact. I’m checking it out :

[ServeurpersoCom]

OpenAI/Anthropic WebUI, the quality target :

First try :

I can do better !

[ServeurpersoCom]

Need better !

[ServeurpersoCom]

Done :

Markdown -> all link in new tab -> OK
Markdown -> OK :

LaTeX -> OK :

[zzokkolma]

I tested your PR and it does seem to solve the "missing HTML tokens" problem. I wasn't able to reproduce it with your changes.
The reason I mentioned the other issue is because I ran into missing non-HTML output tokens once, suggesting there are other lingering problems. The HTML part however your PR seems to solve.

[ServeurpersoCom]

I tested your PR and it does seem to solve the "missing HTML tokens" problem. I wasn't able to reproduce it with your changes. The reason I mentioned the other issue is because I ran into missing non-HTML output tokens once, suggesting there are other lingering problems. The HTML part however your PR seems to solve.

Thanks a lot for testing and confirming the fix !
For the remaining issue where non-HTML tokens occasionally disappear, it would really help to get a bit more context so I can reproduce and instrument it properly.

If it happens again, could you please:

grab a quick screenshot showing roughly where the output stops (even a quick draft on Paint is perfect),

mention the exact .gguf model used,

copy your startup command line (with all parameters),

and the prompt text that triggered the loss.

That will let me script repeated runs on my local server and see exactly where the token stream desynchronizes.
Some highly quantized models occasionally inject invisible or non-UTF8 characters (e.g. stray CJK or control tokens), which might explain sporadic truncation on the client side.

Thanks again for taking the time to test, your feedback really helps polish these edge cases.