Eval bug: unknown model architecture: 'qwen_image'

[asinwang]

Name and Version

build: 6106 (5fd160b) with cc (Debian 10.2.1-6) 10.2.1 20210110 for x86_64-linux-gnu
main: llama backend init
main: load the model and apply lora adapter, if any
llama_model_loader: loaded meta data with 3 key-value pairs and 1933 tensors from qwen-image-Q8_0.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv 0: general.architecture str = qwen_image
llama_model_loader: - kv 1: general.quantization_version u32 = 2
llama_model_loader: - kv 2: general.file_type u32 = 7
llama_model_loader: - type f32: 1087 tensors
llama_model_loader: - type q8_0: 840 tensors
llama_model_loader: - type bf16: 6 tensors
print_info: file format = GGUF V3 (latest)
print_info: file type = Q8_0
print_info: file size = 20.27 GiB (8.52 BPW)
llama_model_load: error loading model: error loading model architecture: unknown model architecture: 'qwen_image'
llama_model_load_from_file_impl: failed to load model

Operating systems

Linux

GGML backends

CUDA

Hardware

RTX4090

Models

No response

Problem description & steps to reproduce

./llama-cli -m qwen-image-Q8_0.gguf -i

First Bad Commit

No response

Relevant log output

llama_model_load: error loading model: error loading model architecture: unknown model architecture: 'qwen_image'
llama_model_load_from_file_impl: failed to load model

qwen-image-Q8_0.gguf source：
https://huggingface.co/city96/Qwen-Image-gguf/blob/main/qwen-image-Q8_0.gguf

[iamlemec]

That looks like an image gen model for use with ComfyUI and similar. They also use GGUF for model storage and quantization.

[woheller69]

you cannot run qwen image with llama.cpp

[pwilkin]

You need a custom llama.cpp version/patch from City96 to quantize the diffusion models.

See instructions here:

https://github.com/city96/ComfyUI-GGUF/tree/main/tools

[pwilkin]

Also, City96 already made quants of Qwen-Image:

https://huggingface.co/city96/Qwen-Image-gguf

[asinwang]

Also, City96 already made quants of Qwen-Image:

https://huggingface.co/city96/Qwen-Image-gguf

yes，qwen-image-Q8_0.gguf source https://huggingface.co/city96/Qwen-Image-gguf/blob/main/qwen-image-Q8_0.gguf
Does llama.cpp support running?
I need an image to image service program

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale.