Qwen-Image-2.1 ncnn Vulkan

❗ ❗ ❗ This software is in the early development stage, it may bite your cat

ncnn implementation of Qwen-Image-2.1, with text-to-image, image editing, multi-reference image input and transparent RGBA output.

qwenimage-ncnn-vulkan uses ncnn project as the universal neural network inference framework.

Download Windows/Linux/macOS Executable for Intel/AMD/NVIDIA/Apple-Silicon GPU

https://github.com/nihui/qwenimage-ncnn-vulkan/releases

This package includes all the binaries required. It is portable, so no CUDA, PyTorch or Python runtime environment is needed :)

Download the qwenimage21 model folder:

https://huggingface.co/nihui-szyl/qwen-image-ncnn

Put it under the models/ directory when running from the source tree, or pass its path with -m.

qwenimage-ncnn-vulkan
models/
    qwenimage21/
        processor/
            vocab.txt
            merges.txt
        text_encoder/
            text_encoder.ncnn.param
            text_encoder.ncnn.bin
        vision/
            vision_encoder.ncnn.param
            vision_encoder.ncnn.bin
            vision_pos_embed.f32
        transformer/
            input.ncnn.param
            input.ncnn.bin
            blocks.ncnn.param
            blocks.ncnn.bin
            output.ncnn.param
            output.ncnn.bin
        vae/
            encoder.ncnn.param
            encoder.ncnn.bin
            decoder.ncnn.param
            decoder.ncnn.bin

One model package supports text-to-image and image editing, dynamic output sizes and up to ten reference images.

Qwen-Image-2.1: A unified text-to-image generation and image editing model open-sourced by the Qwen team. With a 7B parameter visual component across 32 Single-Stream DiT layers, it balances generation quality, inference efficiency, and versatility.

https://github.com/QwenLM/Qwen-Image-2.1

• Minimum (Linux / macOS): 16GB RAM, any Vulkan capable GPU

• Minimum (Windows):

  Due to WDDM limitations: Vulkan applications can only use half of the system RAM.

  The following condition shall be met: (Half of system RAM) + (GPU memory) >= 16GB

  Examples of valid combinations:

  • Any amount RAM, 16GB dedicated GPU
  • 16GB RAM, 8GB dedicated GPU
  • 24GB RAM, 4GB dedicated GPU
  • 32GB RAM, any Vulkan capable GPU

• Recommended: 32GB RAM, 16GB dedicated GPU with tensorcore/matrix hardware

• Low-VRAM mode and VAE encoder/decoder tiling adapt automatically to the available GPU memory at the start of each generation

• CPU inference is available with -g -1

Earlier text-to-image memory measurements before prefix KV caching (RTX 3060, BF16, 2 steps):

These historical measurements are examples, not current minimum VRAM guarantees. Current memory requirements also depend on prompt length and KV caches. Tile sizes are selected automatically from the current memory budget and may be larger when more memory is available. No manual low-VRAM or tile setting is required.

The peaks are VRAM increments over the usage before the process starts. Allow additional GPU memory for the driver, desktop and other applications when choosing a graphics card.

Image editing also accounts for reference image sizes and negative prompts when choosing its memory policy. When needed, reference KV caches are kept in system RAM and transferred one Transformer block at a time. More reference images require more system RAM and may take longer to process. VAE encoder and decoder tile sizes are selected independently. If the estimated Transformer workspace or full-image VAE attention cannot fit even with offloading and tiling, generation stops with an insufficient-memory error; reduce the output or reference image sizes, or use fewer references.

Text to image

qwenimage-ncnn-vulkan -p "A small red kite over a quiet lake." -o output.png

Text to image with a negative prompt

qwenimage-ncnn-vulkan -p "A small red kite over a quiet lake." -n "blurry image, dull colors" -w 4 -o output.png

The -w value maps to Torch true_cfg_scale and defaults to 1.0. The negative prompt participates in CFG only when -w is greater than 1.

Image editing

qwenimage-ncnn-vulkan -i input.png -p "Change the clothes to a blue jacket." -o output.png

Multiple reference images

qwenimage-ncnn-vulkan -i subject.png -i clothing.png -i background.png -p "Combine these references into one coherent image." -o output.png

Transparent image generation

qwenimage-ncnn-vulkan -p "A glass bottle on a transparent background." -o output.png

Set image size, denoise steps, seed and GPU

qwenimage-ncnn-vulkan -s 1024,1024 -l 40 -r 42 -g 0 -p "A red flower." -o output.png

Batch generation

qwenimage-ncnn-vulkan -b 4 -r 42 -p "A red flower." -o output.png

With -b greater than one, outputs are saved as output-0.png, output-1.png, and so on. Each image uses the next seed value. Use -i once for each reference image, up to ten images. Text-to-image sizes must be multiples of 16; image-editing sizes must be multiples of 32.

The default output is an RGBA PNG. A jpg or jpeg suffix writes RGB JPEG output, and a webp suffix writes lossless RGBA WebP output. PNG and WebP alpha can be used for transparent image generation.

Usage: qwenimage-ncnn-vulkan -p prompt -o outfile [options]...

  -h                   show this help
  -p prompt            prompt
  -n negative-prompt   negative prompt (optional)
  -w guidance-scale    true CFG scale (default=1.0)
  -o output-path       output image path (default=out.png)
  -i input-image       reference image for editing (repeat 1 to 10 times)
  -s image-size        image resolution (default=1024,1024)
  -l steps             denoise steps (default=40)
  -r random-seed       random seed (default=42)
  -m model-path        qwen-image model path (default=models/qwenimage21)
  -g gpu-id            GPU device to use (-1=cpu, default=auto)
  -b batch-size        batched generation (default=1)

If you encounter a crash or error, try upgrading your GPU driver:

• Intel: https://downloadcenter.intel.com/product/80939/Graphics-Drivers
• AMD: https://www.amd.com/en/support
• NVIDIA: https://www.nvidia.com/Download/index.aspx

1. Clone this project with all submodules

git clone https://github.com/nihui/qwenimage-ncnn-vulkan.git
cd qwenimage-ncnn-vulkan
git submodule update --init --recursive --depth 1

2. Build with CMake

mkdir build
cd build
cmake ../src
cmake --build . -j 4

TBA

• https://github.com/QwenLM/Qwen-Image-2.1

• https://github.com/Tencent/ncnn for fast neural network inference on ALL PLATFORMS
• https://github.com/futz12/ncnn_llm for BPE tokenizer
• https://github.com/webmproject/libwebp for encoding and decoding Webp images on ALL PLATFORMS
• https://github.com/libjpeg-turbo/libjpeg-turbo for encoding and decoding JPEG images on ALL PLATFORMS
• https://github.com/pnggroup/libpng for encoding and decoding PNG images on ALL PLATFORMS
• https://github.com/zlib-ng/zlib-ng for encoding and decoding PNG images on ALL PLATFORMS
• https://github.com/tronkko/dirent for listing files in directory on Windows