Skip to content

Conversation

@bluebread
Copy link

Make sure to read the contributing guidelines before submitting a PR

@sfallah This bug was because I forgot to add token swapping logic... sorry for that. The decoding should work now. FYI here are the result and my test settings.

image

@sfallah
Copy link
Owner

sfallah commented Nov 24, 2025

@bluebread
no problem,
While debugging, I tried some exising (gemma3) and ran to the "marker" issue with old working models.
My fix locally (not committed yet) changes for fixing the issue with existing models is below.
Can you please have a look at, and please test with som existing models.

Screenshot 2025-11-24 at 09 35 35

I am e.g. using:

build/bin/llama-mtmd-cli \
-hf ggml-org/Qwen2.5-VL-3B-Instruct-GGUF:Q4_K_M \
--image tools/mtmd/test-1.jpeg \
-p "what is the publisher name of the newspaper?" \
--temp 0 -n 128

@bluebread
Copy link
Author

@sfallah Fixed it. Will be more careful next time...

image

@sfallah sfallah merged commit 8810940 into sfallah:sf/deepseek-ocr Nov 24, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants