model: add Janus Pro for image understanding

[ravenouse]

This pull request introduces support for the Janus‑Pro 1B and Janus‑Pro 7B models within the llama.cpp framework.

The focus of this update is on image understanding (i.e., visual-input → textual or conceptual output).
Image generation is not covered by this PR.

Usage & Current Progress

Convert models to GGUF files:

# Convert the base Janus-Pro 1B model
python convert_hf_to_gguf.py deepseek-community/Janus-Pro-1B \
    --remote \
    --outfile janus-pro-1b-f16.gguf \
    --outtype f16

# Convert the mmproj component
python convert_hf_to_gguf.py deepseek-community/Janus-Pro-1B \
    --remote \
    --outfile mmproj-janus-pro-1b-f16.gguf \
    --outtype f16 \
    --mmproj

The converted GGUF files can be accessed here: https://huggingface.co/Ericwang/Janus-Pro-1B-GGUF

Run the model:

# Build the project:
cmake -B build
cmake --build build --target llama-mtmd-cli

./build/bin/llama-mtmd-cli \
    -m janus-pro-1b-f16.gguf \
    --mmproj mmproj-janus-pro-1b-f16.gguf \
    --chat-template deepseek

References

Janus-Pro 1B model card:
https://huggingface.co/deepseek-community/Janus-Pro-1B

Janus-Pro 7B model card:
https://huggingface.co/deepseek-community/Janus-Pro-7B

Configurations:
https://huggingface.co/deepseek-community/Janus-Pro-1B/blob/main/config.json
https://huggingface.co/deepseek-community/Janus-Pro-7B/blob/main/config.json

HF Implementation:
https://github.com/huggingface/transformers/tree/main/src/transformers/models/janus

[ravenouse]

Tested with this image: https://www.pixelstalk.net/wp-content/uploads/2016/04/Golden-retriever-dogs-high-definition-wallpapers.jpg

[ravenouse]

Hi @CISC and @ngxson ,

Thank you for the thorough review and valuable feedback.
I've addressed all the comments. I also re-ran the conversion and inference workflows, and both are working as expected.

Ready for another look when you have a moment. Thanks a lot!

[ravenouse]

Thanks again for the review!

Just updated the code and tested it again.

I've updated the code and retested it with the following image:
https://1.bp.blogspot.com/-tLB0HRLcOp4/Tj4Pvhsq6vI/AAAAAAAAAG8/h6ahy6g4GJI/s1600/Llama_lying_down.jpg

PS: The forced push was made to correct a formatting typo in the commit message.

[ngxson]

Looking good! Merging once the CI passes

[CISC]

Fix all the whitespace errors though. :)
https://github.com/ggml-org/llama.cpp/actions/runs/19016845485/job/54305935425?pr=16906

[ravenouse]

Hi @CISC and @ngxson ,

Thank you so much for the quick review and support to finalize this PR. Truly appreciated!