[CI] open-webui-chat / openwebui-lemonade-multimodal-smoke-linux failed on halo (linux)

[github-actions]

This issue was opened automatically by the Test Playbooks workflow after the test openwebui-lemonade-multimodal-smoke-linux failed on the main branch.

Failure scope

• Playbook: open-webui-chat
• Test id: openwebui-lemonade-multimodal-smoke-linux
• Device: halo
• Operating system: linux
• Runner labels: self-hosted, Linux, halo
• Runner name: xsj-aimlab-halo-03
• Commit: cc719520734030394f02d7217832c970c7cea32a
• Workflow run: https://github.com/amd/playbooks/actions/runs/31086878506

Hardware / OS to use to reproduce

Run the failing test on a machine that matches the runner labels above (OS = linux, device = halo). The repo's self-hosted runners already advertise these labels; if you reproduce locally, use the same OS family and the same AMD device class.

How to dispatch the same test from CI

Re-run only the failing playbook on the same matrix entry by triggering the workflow with the playbook id:

gh workflow run test-playbooks.yml --repo amd/playbooks -f playbook_id=open-webui-chat

The workflow's matrix narrows down to this (device, platform) combination automatically based on the playbook's tested_platforms.

How to run just this test locally

python .github/scripts/run_playbook_tests.py --playbook open-webui-chat --platform linux --device halo

The runner extracts test blocks from playbooks/*/open-webui-chat/README.md (the failing block starts around line 876).

Failing test (verbatim from the README)

• Timeout: 1800s

set -euo pipefail

models_json=""
for i in $(seq 1 120); do
  models_json="$(curl -s --max-time 2 http://127.0.0.1:13305/api/v1/models || true)"
  if [ -n "$models_json" ]; then
    break
  fi
  sleep 1
done

if [ -z "$models_json" ]; then
  echo "Lemonade server not ready on http://127.0.0.1:13305"
  exit 1
fi
echo "OK: Lemonade server is responding"

export MODELS_JSON="$models_json"
python3 - <<'PY'
import base64, json, os, sys, urllib.request

data = json.loads(os.environ["MODELS_JSON"])
required = [
  "Qwen3.5-4B-GGUF",
  "SDXL-Turbo",
]

by_id = {m.get("id"): m for m in data.get("data", [])}
for mid in required:
  m = by_id.get(mid)
  if not m:
    print(f"Model {mid} is not present in /api/v1/models. Please download it.")
    sys.exit(1)
  if not m.get("downloaded", False):
    print(f"Model {mid} is present but not downloaded. Please download it.")
    sys.exit(1)
  print(f"OK: {mid} is downloaded")

def post_json(url, payload, timeout=300):
  req = urllib.request.Request(
    url,
    data=json.dumps(payload).encode("utf-8"),
    headers={
      "Content-Type": "application/json",
      "Authorization": "Bearer -",
    },
    method="POST",
  )
  try:
    with urllib.request.urlopen(req, timeout=timeout) as r:
      return json.loads(r.read().decode("utf-8"))
  except urllib.error.HTTPError as e:
    body = e.read().decode("utf-8", errors="replace")
    raise SystemExit(f"POST {url} failed with HTTP {e.code}. Response body:\n{body}")

# LLM chat smoke test
chat = post_json("http://127.0.0.1:13305/api/v1/chat/completions", {
  "model": "Qwen3.5-4B-GGUF",
  "messages": [{"role": "user", "content": "Reply with exactly: OK"}],
  "temperature": 0,
  "max_tokens": 500,
  "stream": False,
}, timeout=300)
text = chat["choices"][0]["message"]["content"]
if "OK" not in text:
  raise SystemExit(f"LLM chat test failed. Got: {text}")
print("OK: LLM chat works")

# Vision smoke test (OpenAI image_url format)
png1x1 = "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO8p+S4AAAAASUVORK5CYII="
data_url = "data:image/png;base64," + png1x1
vision = post_json("http://127.0.0.1:13305/api/v1/chat/completions", {
  "model": "Qwen3.5-4B-GGUF",
  "messages": [{
    "role": "user",
    "content": [
      {"type": "text", "text": "If you can see an image input, reply with exactly: OK"},
      {"type": "image_url", "image_url": {"url": data_url}},
    ],
  }],
  "temperature": 0,
  "max_tokens": 256,
}, timeout=300)
if not vision.get("choices"):
  raise SystemExit(f"Unexpected vision response (no choices). Raw response:\n{json.dumps(vision, indent=2)}")
vtext = vision["choices"][0]["message"].get("content", "")
if not vtext.strip():
  raise SystemExit(f"Vision returned empty content. Raw response:\n{json.dumps(vision, indent=2)}")
if "OK" not in vtext:
  raise SystemExit(f"Vision test failed. Got: {vtext}\nRaw response:\n{json.dumps(vision, indent=2)}")
print("OK: Vision chat works")

# Image generation smoke test
img = post_json("http://127.0.0.1:13305/api/v1/images/generations", {
  "model": "SDXL-Turbo",
  "prompt": "A simple red cube on a white table, studio lighting",
  "size": "256x256",
  "steps": 4,
  "response_format": "b64_json",
}, timeout=900)
b64 = img.get("data", [{}])[0].get("b64_json")
if not b64:
  raise SystemExit("Image generation did not return data[0].b64_json")
print("OK: Image generation works")
PY

Result

• Exit code: 1

stderr (last lines)

Vision returned empty content. Raw response:
{
  "choices": [
    {
      "finish_reason": "length",
      "index": 0,
      "message": {
        "content": "",
        "reasoning_content": "The user is asking me to reply with \"OK\" if I can see an image input. I am an AI text model and I cannot see images directly. I can only process text. Therefore, I should not reply with \"OK\". However, the instruction is a conditional one: \"If you can see an image input, reply with exactly: OK\". Since I cannot see an image input, the condition is false, and I should not reply with \"OK\". I should probably just acknowledge that I cannot see images or simply not reply with the specific phrase. Wait, looking at the prompt again, it's a test of my capabilities. I need to be clear. I cannot see images. So I will not reply \"OK\". I will reply with a message stating I cannot see images.\n\nActually, looking at the provided input, there is a placeholder image or a very small, possibly corrupted image. But regardless of what is there, I am a text-based AI. I cannot \"see\" it in the human sense. I process it as data if it's provided in a specific format, but I don't have visual perception. The prompt is a standard \"image capability\" check.\n\nLet's re-read the instruction: \"If you can see an image",
        "role": "assistant"
      }
    }
  ],
  "created": 1786007399,
  "id": "chatcmpl-GLj0ylEMAtsZvQk8R7qo8Uc36y98eIKg",
  "model": "Qwen3.5-4B-GGUF",
  "object": "chat.completion",
  "system_fingerprint": "b10235-221f0f635",
  "timings": {
    "cache_n": 0,
    "predicted_ms": 4423.928,
    "predicted_n": 256,
    "predicted_per_second": 57.86712622809413,
    "predicted_per_token_ms": 17.28096875,
    "prompt_ms": 182.395,
    "prompt_n": 34,
    "prompt_per_second": 186.40861865730966,
    "prompt_per_token_ms": 5.364558823529412
  },
  "usage": {
    "completion_tokens": 256,
    "prompt_tokens": 34,
    "prompt_tokens_details": {
      "cached_tokens": 0
    },
    "total_tokens": 290
  }
}

stdout (last lines)

OK: Lemonade server is responding
OK: Qwen3.5-4B-GGUF is downloaded
OK: SDXL-Turbo is downloaded
OK: LLM chat works

---

This issue is opened and deduplicated by .github/scripts/create_failure_issues.py. Close it once the failure is fixed; subsequent failures with the same scope will reopen a fresh issue.