[CI] open-webui-chat / openwebui-lemonade-multimodal-smoke-linux failed on r9700 (linux)

[github-actions]

This issue was opened automatically by the Test Playbooks workflow after the test openwebui-lemonade-multimodal-smoke-linux failed on the main branch.

Failure scope

• Playbook: open-webui-chat
• Test id: openwebui-lemonade-multimodal-smoke-linux
• Device: r9700
• Operating system: linux
• Runner labels: self-hosted, Linux, r9700
• Runner name: MNB-UCICD-DT457
• Commit: cc719520734030394f02d7217832c970c7cea32a
• Workflow run: https://github.com/amd/playbooks/actions/runs/31086878506

Hardware / OS to use to reproduce

Run the failing test on a machine that matches the runner labels above (OS = linux, device = r9700). The repo's self-hosted runners already advertise these labels; if you reproduce locally, use the same OS family and the same AMD device class.

How to dispatch the same test from CI

Re-run only the failing playbook on the same matrix entry by triggering the workflow with the playbook id:

gh workflow run test-playbooks.yml --repo amd/playbooks -f playbook_id=open-webui-chat

The workflow's matrix narrows down to this (device, platform) combination automatically based on the playbook's tested_platforms.

How to run just this test locally

python .github/scripts/run_playbook_tests.py --playbook open-webui-chat --platform linux --device r9700

The runner extracts test blocks from playbooks/*/open-webui-chat/README.md (the failing block starts around line 876).

Failing test (verbatim from the README)

• Timeout: 1800s

set -euo pipefail

models_json=""
for i in $(seq 1 120); do
  models_json="$(curl -s --max-time 2 http://127.0.0.1:13305/api/v1/models || true)"
  if [ -n "$models_json" ]; then
    break
  fi
  sleep 1
done

if [ -z "$models_json" ]; then
  echo "Lemonade server not ready on http://127.0.0.1:13305"
  exit 1
fi
echo "OK: Lemonade server is responding"

export MODELS_JSON="$models_json"
python3 - <<'PY'
import base64, json, os, sys, urllib.request

data = json.loads(os.environ["MODELS_JSON"])
required = [
  "Qwen3.5-4B-GGUF",
  "SDXL-Turbo",
]

by_id = {m.get("id"): m for m in data.get("data", [])}
for mid in required:
  m = by_id.get(mid)
  if not m:
    print(f"Model {mid} is not present in /api/v1/models. Please download it.")
    sys.exit(1)
  if not m.get("downloaded", False):
    print(f"Model {mid} is present but not downloaded. Please download it.")
    sys.exit(1)
  print(f"OK: {mid} is downloaded")

def post_json(url, payload, timeout=300):
  req = urllib.request.Request(
    url,
    data=json.dumps(payload).encode("utf-8"),
    headers={
      "Content-Type": "application/json",
      "Authorization": "Bearer -",
    },
    method="POST",
  )
  try:
    with urllib.request.urlopen(req, timeout=timeout) as r:
      return json.loads(r.read().decode("utf-8"))
  except urllib.error.HTTPError as e:
    body = e.read().decode("utf-8", errors="replace")
    raise SystemExit(f"POST {url} failed with HTTP {e.code}. Response body:\n{body}")

# LLM chat smoke test
chat = post_json("http://127.0.0.1:13305/api/v1/chat/completions", {
  "model": "Qwen3.5-4B-GGUF",
  "messages": [{"role": "user", "content": "Reply with exactly: OK"}],
  "temperature": 0,
  "max_tokens": 500,
  "stream": False,
}, timeout=300)
text = chat["choices"][0]["message"]["content"]
if "OK" not in text:
  raise SystemExit(f"LLM chat test failed. Got: {text}")
print("OK: LLM chat works")

# Vision smoke test (OpenAI image_url format)
png1x1 = "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO8p+S4AAAAASUVORK5CYII="
data_url = "data:image/png;base64," + png1x1
vision = post_json("http://127.0.0.1:13305/api/v1/chat/completions", {
  "model": "Qwen3.5-4B-GGUF",
  "messages": [{
    "role": "user",
    "content": [
      {"type": "text", "text": "If you can see an image input, reply with exactly: OK"},
      {"type": "image_url", "image_url": {"url": data_url}},
    ],
  }],
  "temperature": 0,
  "max_tokens": 256,
}, timeout=300)
if not vision.get("choices"):
  raise SystemExit(f"Unexpected vision response (no choices). Raw response:\n{json.dumps(vision, indent=2)}")
vtext = vision["choices"][0]["message"].get("content", "")
if not vtext.strip():
  raise SystemExit(f"Vision returned empty content. Raw response:\n{json.dumps(vision, indent=2)}")
if "OK" not in vtext:
  raise SystemExit(f"Vision test failed. Got: {vtext}\nRaw response:\n{json.dumps(vision, indent=2)}")
print("OK: Vision chat works")

# Image generation smoke test
img = post_json("http://127.0.0.1:13305/api/v1/images/generations", {
  "model": "SDXL-Turbo",
  "prompt": "A simple red cube on a white table, studio lighting",
  "size": "256x256",
  "steps": 4,
  "response_format": "b64_json",
}, timeout=900)
b64 = img.get("data", [{}])[0].get("b64_json")
if not b64:
  raise SystemExit("Image generation did not return data[0].b64_json")
print("OK: Image generation works")
PY

Result

• Exit code: 1

stderr (last lines)

Vision returned empty content. Raw response:
{
  "choices": [
    {
      "finish_reason": "length",
      "index": 0,
      "message": {
        "content": "",
        "reasoning_content": "The user is asking me to confirm if I can see an image input.\n1.  **Analyze the input:** The user provided an image. It appears to be a blank white square or a very faint, low-resolution image.\n2.  **Check capabilities:** I am an AI text model. I cannot \"see\" images in the human sense, but I can process image inputs if they are provided in the context. However, in this specific interaction, the user is asking a conditional question: \"If you can see an image input, reply with exactly: OK\".\n3.  **Evaluate the image:** The image provided is just a blank white square. It contains no discernible content.\n4.  **Formulate response:** Since the user is asking me to reply with \"OK\" *if* I can see an image input, and there is indeed an image input (even if it's blank), the correct response is to follow the instruction. Wait, looking closely at the prompt structure, it's a standard \"image recognition\" check. The user is testing if I can process an image.\n5.  **Re-evaluating the \"image\":** The image provided is actually just a blank white box. It's likely",
        "role": "assistant"
      }
    }
  ],
  "created": 1786008011,
  "id": "chatcmpl-frSsEnboddUrJUx1hFprcSHSBrfjcMjW",
  "model": "Qwen3.5-4B-GGUF",
  "object": "chat.completion",
  "system_fingerprint": "b10241-9bd4c09ea",
  "timings": {
    "cache_n": 0,
    "predicted_ms": 2028.371,
    "predicted_n": 256,
    "predicted_per_second": 126.20965296782491,
    "predicted_per_token_ms": 7.92332421875,
    "prompt_ms": 151.964,
    "prompt_n": 34,
    "prompt_per_second": 223.73720091600643,
    "prompt_per_token_ms": 4.4695294117647055
  },
  "usage": {
    "completion_tokens": 256,
    "prompt_tokens": 34,
    "prompt_tokens_details": {
      "cached_tokens": 0
    },
    "total_tokens": 290
  }
}

stdout (last lines)

OK: Lemonade server is responding
OK: Qwen3.5-4B-GGUF is downloaded
OK: SDXL-Turbo is downloaded
OK: LLM chat works

---

This issue is opened and deduplicated by .github/scripts/create_failure_issues.py. Close it once the failure is fixed; subsequent failures with the same scope will reopen a fresh issue.

[github-actions]

Test failed again on a subsequent CI run.

• Workflow run: https://github.com/amd/playbooks/actions/runs/31161490330
• Commit: 61b8d874c4ee8123ae8a5871e81ab190365b9300
• Runner: MNB-UCICD-DT457 (self-hosted, Linux, r9700)