[Bug] prompt weighting produces broken results on Anima

[wbruna]

Git commit

master-596-90e87bc

Operating System & Version

Debian 13

GGML backends

Vulkan, HIP

Command-line arguments used

sd-cli --diffusion-model UltraReal_anima3.safetensors --llm Qwen3-0.6B-UD-Q4_K_XL.gguf -W 512 -H 768 --vae Qwen_Image-VAE.safetensors --steps 30 --cfg-scale 7 --diffusion-fa -p "Raw smartphone photo shot with a wide-angle lens, candid photo of a Ereshkigal (ereshkigal (fate)), with long blonde hair in twintails (twintails) and parted bangs (parted bangs), tied with red ribbons and a bow (red ribbon, hair ribbon, red bow)."

Steps to reproduce

Originally on LostRuins/koboldcpp#2205

A prompt like:

Raw smartphone photo shot with a wide-angle lens, candid photo of a Ereshkigal (ereshkigal (fate)), with long blonde hair in twintails (twintails) and parted bangs (parted bangs), tied with red ribbons and a bow (red ribbon, hair ribbon, red bow).

produces a garbage image on both Vulkan and ROCm:

But just by removing all parenthesis I get:

What you expected to happen

Both prompts producing images

What actually happened

A garbage result when changing token weights.

Logs / error messages / stack trace

tokenizer logs

[DEBUG] conditioner.hpp:1585 - parse 'Raw smartphone photo shot with a wide-angle lens, candid photo of a Ereshkigal (ereshkigal (fate)), with long blonde hair in twintails (twintails) and parted bangs (parted bangs), tied with red ribbons and a bow (red ribbon, hair ribbon, red bow).' to [['Raw smartphone photo shot with a wide-angle lens, candid photo of a Ereshkigal ', 1], ['ereshkigal ', 1.1], ['fate', 1.21], [', with long blonde hair in twintails ', 1], ['twintails', 1.1], [' and parted bangs ', 1], ['parted bangs', 1.1], [', tied with red ribbons and a bow ', 1], ['red ribbon, hair ribbon, red bow', 1.1], ['.', 1], ]
[DEBUG] bpe_tokenizer.cpp:183  - split prompt "Raw smartphone photo shot with a wide-angle lens, candid photo of a Ereshkigal " to tokens ["Raw", "Ġsmartphone", "Ġphoto", "Ġshot", "Ġwith", "Ġa", "Ġwide", "-angle", "Ġlens", ",", "Ġcandid", "Ġphoto", "Ġof", "Ġa", "ĠE", "resh", "k", "ig", "al", "Ġ", ]
[DEBUG] bpe_tokenizer.cpp:183  - split prompt "ereshkigal " to tokens ["er", "esh", "k", "ig", "al", "Ġ", ]
[DEBUG] bpe_tokenizer.cpp:183  - split prompt "fate" to tokens ["f", "ate", ]
[DEBUG] bpe_tokenizer.cpp:183  - split prompt ", with long blonde hair in twintails " to tokens [",", "Ġwith", "Ġlong", "Ġblonde", "Ġhair", "Ġin", "Ġtw", "int", "ails", "Ġ", ]
[DEBUG] bpe_tokenizer.cpp:183  - split prompt "twintails" to tokens ["tw", "int", "ails", ]
[DEBUG] bpe_tokenizer.cpp:183  - split prompt " and parted bangs " to tokens ["Ġand", "Ġparted", "Ġbang", "s", "Ġ", ]
[DEBUG] bpe_tokenizer.cpp:183  - split prompt "parted bangs" to tokens ["part", "ed", "Ġbang", "s", ]
[DEBUG] bpe_tokenizer.cpp:183  - split prompt ", tied with red ribbons and a bow " to tokens [",", "Ġtied", "Ġwith", "Ġred", "Ġrib", "bons", "Ġand", "Ġa", "Ġbow", "Ġ", ]
[DEBUG] bpe_tokenizer.cpp:183  - split prompt "red ribbon, hair ribbon, red bow" to tokens ["red", "Ġribbon", ",", "Ġhair", "Ġribbon", ",", "Ġred", "Ġbow", ]
[DEBUG] bpe_tokenizer.cpp:183  - split prompt "." to tokens [".", ]
[DEBUG] t5_unigram_tokenizer.cpp:336  - split prompt "Raw smartphone photo shot with a wide-angle lens, candid photo of a Ereshkigal " to tokens ["▁Raw", "▁smartphone", "▁photo", "▁shot", "▁with", "▁", "a", "▁wide", "-", "angle", "▁lens", ",", "▁candid", "▁photo", "▁of", "▁", "a", "▁Er", "e", "s", "h", "ki", "gal", ]
[DEBUG] t5_unigram_tokenizer.cpp:336  - split prompt "ereshkigal " to tokens ["▁", "er", "e", "s", "h", "ki", "gal", ]
[DEBUG] t5_unigram_tokenizer.cpp:336  - split prompt "fate" to tokens ["▁fate", ]
[DEBUG] t5_unigram_tokenizer.cpp:336  - split prompt ", with long blonde hair in twintails " to tokens ["▁", ",", "▁with", "▁long", "▁blonde", "▁hair", "▁in", "▁twin", "tail", "s", ]
[DEBUG] t5_unigram_tokenizer.cpp:336  - split prompt "twintails" to tokens ["▁twin", "tail", "s", ]
[DEBUG] t5_unigram_tokenizer.cpp:336  - split prompt " and parted bangs " to tokens ["▁", "▁and", "▁parte", "d", "▁ban", "g", "s", ]
[DEBUG] t5_unigram_tokenizer.cpp:336  - split prompt "parted bangs" to tokens ["▁parte", "d", "▁ban", "g", "s", ]
[DEBUG] t5_unigram_tokenizer.cpp:336  - split prompt ", tied with red ribbons and a bow " to tokens ["▁", ",", "▁tied", "▁with", "▁red", "▁ribbon", "s", "▁and", "▁", "a", "▁bow", ]
[DEBUG] t5_unigram_tokenizer.cpp:336  - split prompt "red ribbon, hair ribbon, red bow" to tokens ["▁red", "▁ribbon", ",", "▁hair", "▁ribbon", ",", "▁red", "▁bow", ]
[DEBUG] t5_unigram_tokenizer.cpp:336  - split prompt "." to tokens ["▁", ".", ]

Additional context / environment details

No response

[TkskKurumi]

I've encountered similar issue on Ubuntu + CUDA (RTX 3080, Driver Version: 590.48.01 CUDA Version: 13.1)

DiT: https://huggingface.co/circlestone-labs/Anima/blob/main/split_files/diffusion_models/anima-preview3-base.safetensors

launching script

/home/tkskkurumi/stable-diffusion.cpp/build_90e87bc/bin/sd-server \
  --diffusion-model /home/tkskkurumi/LLM/download/Anima/anima-preview3.safetensors \
  --vae "/home/tkskkurumi/LLM/download/Anima/qwen_image_vae.safetensors" \
  --llm "/home/tkskkurumi/LLM/download/Anima/qwen3_06b_base.safetensors" \
  --lora-model-dir /home/tkskkurumi/LLM/download/Anima/loras \
  --steps 40 \
  --negative-prompt "censored, bar censor, mosaic censor, blur censor, mosaic censoring, fused fingers, broken fingers, extra digits, missing fingers, distorted, distortion, signature, watermark, blur, blurry, overexposure, sketch, draft, unfinished, blackout, censored,  messy, out of focus, bad anatomy, bad proportion, image noise, lossy compression, low quality, worst quality, score_1, score_2, obese" \
  --fa \
  --vae-tiling \
  --vae-tile-size 64x64 \
  --width 800 \
  --height 1152 \
  -l 0.0.0.0 \
  --listen-port 8100 \
  --cfg-scale 5 \
  --offload-to-cpu \
  --sampling-method er_sde \
  -v