-
Notifications
You must be signed in to change notification settings - Fork 14.1k
Description
Name and Version
ggml_cuda_init: found 1 CUDA devices:
Device 0: NVIDIA GeForce RTX 4060 Laptop GPU, compute capability 8.9, VMM: yes
version: 6907 (bea0452)
built with MSVC 19.44.35219.0 for x64
Operating systems
Windows
GGML backends
HIP, CUDA
Hardware
4060 or radeon 8060S
Models
No response
Problem description & steps to reproduce
llama-server -m obsidian-q6.gguf --mmproj mmproj-obsidian-f16.gguf
https://huggingface.co/NousResearch/Obsidian-3B-V0.5-GGUF
clip_init: failed to load model 'mmproj-obsidian-f16.gguf': load_hparams: unknown projector type:
mtmd_init_from_file: error: Failed to load CLIP model from mmproj-obsidian-f16.gguf
srv load_model: failed to load multimodal model, 'mmproj-obsidian-f16.gguf'
srv operator(): operator(): cleaning up before exit...
main: exiting due to model loading error
I have go through https://github.com/ggml-org/llama.cpp?tab=readme-ov-file#multimodal
Looks below 4 models failed with same issue
https://huggingface.co/advanced-stack/bakllava-mistral-v1-gguf/
https://huggingface.co/NousResearch/Obsidian-3B-V0.5-GGUF/tree/main
https://huggingface.co/nakodanei/ShareGPT4V-13B_GGUF/
https://huggingface.co/cmp-nct/Yi-VL-6B-GGUF
First Bad Commit
No response
Relevant log output
clip_init: failed to load model 'mmproj-obsidian-f16.gguf': load_hparams: unknown projector type:
mtmd_init_from_file: error: Failed to load CLIP model from mmproj-obsidian-f16.gguf
srv load_model: failed to load multimodal model, 'mmproj-obsidian-f16.gguf'
srv operator(): operator(): cleaning up before exit...
main: exiting due to model loading error