RDNA2

[midnightx86]

Hi, i would like to ask if could be added gfx1032 to the list of working architectures.
Have a nice day!

[VecSzn]

Hi! gfx1032 (RX 6600 / 6650 XT) isn't on the list yet because nobody has run it. Setup knows two RDNA2 chips today, gfx1030 (RX 6800 / 6900) and gfx1031 (RX 6700 XT), both marked unvalidated; they use AMD's ROCm wheels for the gfx103X family, and gfx1032 would be added the same way. See docs/AMD_HIP.md.

Two questions so I can help:

1. Which OS, card and VRAM? AMD cards on Windows run Qwen3.8-Flash-Next only; the models PolyStrata adds run on AMD under Linux.
2. Would you test it if I add gfx1032 to setup? A first run and the engine log are all I'd need.

Until then, on Linux there's a common way to run an RX 6600 on ROCm: HSA_OVERRIDE_GFX_VERSION=10.3.0 makes ROCm treat the card as a gfx1030, so the gfx1030 build runs on it. I haven't tried it with PolyStrata, so I can't promise it works, but if you try it, a post here with what happened would help.

[midnightx86]

Im running the gfx1032 on my system, i did with help of ai as im not a developer.
Im using Linux Fedora, i have a rx 6600 xt with 8Gb vram, im running Qwen 3.6 35b a3b.
67.6t/s - Decode last request
434t/s - Prefill last request
Im looking for that log, but i cannot find the one with my gpu information, and yes, if you do for the gfx1032 im able to try it.
Thank you very much! appreciate it!
Just in case is this log what you need, i write it here:
MidNight@fedora:~/PolyStrata$ ./run-qwen36-ud-q4_k_m.sh
loading the model (the first start takes a minute or two) ...
[strata] starting the engine: reading the model's weights ...
[strata] thinking level: none (reasoning_effort in the config; a request, or the Chat settings shared with other apps, can set its own)
[strata] also answers to the host names host.containers.internal (allowed_hosts)
ready: http://127.0.0.1:8080/v1 (OpenAI: /v1/chat/completions, Anthropic: /v1/messages, context 32768 tokens, API key required)
open http://127.0.0.1:8080/ in a browser to chat; close this window to stop the model
from other devices: http://192.168.1.171:8080/ (API: http://192.168.1.171:8080/v1)
[strata] reading the prompt: 86 of 173 tokens, 1 s so far
[strata] thinking: 18 of max 32587 tokens, 20.8 tok/s, 2 s
[strata] thinking: 36 of max 32587 tokens, 18.4 tok/s, 3 s
[strata] thinking: 56 of max 32587 tokens, 18.8 tok/s, 4 s
[strata] thinking: 89 of max 32587 tokens, 22.1 tok/s, 5 s
[strata] thinking: 120 of max 32587 tokens, 23.1 tok/s, 7 s
[strata] thinking: 154 of max 32587 tokens, 24.8 tok/s, 8 s
[strata] thinking: 184 of max 32587 tokens, 25.5 tok/s, 9 s
[strata] thinking: 227 of max 32587 tokens, 27.5 tok/s, 10 s
[strata] thinking: 269 of max 32587 tokens, 28.8 tok/s, 11 s
[strata] thinking: 321 of max 32587 tokens, 31.0 tok/s, 12 s
[strata] thinking: 366 of max 32587 tokens, 32.1 tok/s, 13 s
[strata] thinking: 435 of max 32587 tokens, 35.1 tok/s, 14 s
[strata] thinking: 493 of max 32587 tokens, 36.7 tok/s, 15 s
[strata] thinking: 535 of max 32587 tokens, 37.1 tok/s, 16 s
[strata] thinking: 584 of max 32587 tokens, 37.7 tok/s, 17 s
[strata] thinking: 640 of max 32587 tokens, 38.7 tok/s, 18 s
[strata] thinking: 692 of max 32587 tokens, 39.4 tok/s, 19 s
[strata] thinking: 743 of max 32587 tokens, 40.0 tok/s, 20 s
[strata] thinking: 806 of max 32587 tokens, 41.2 tok/s, 21 s
[strata] thinking: 866 of max 32587 tokens, 42.0 tok/s, 22 s
[strata] thinking: 932 of max 32587 tokens, 43.1 tok/s, 23 s
[strata] thinking: 991 of max 32587 tokens, 43.7 tok/s, 24 s
[strata] thinking: 1043 of max 32587 tokens, 44.0 tok/s, 25 s
[strata] thinking: 1092 of max 32587 tokens, 44.2 tok/s, 26 s
[strata] thinking: 1152 of max 32587 tokens, 44.8 tok/s, 27 s
[strata] thinking: 1215 of max 32587 tokens, 45.5 tok/s, 28 s
[strata] thinking: 1286 of max 32587 tokens, 46.4 tok/s, 29 s
[strata] thinking: 1354 of max 32587 tokens, 47.1 tok/s, 30 s
[strata] thinking: 1415 of max 32587 tokens, 47.5 tok/s, 31 s
[strata] answering: 1475 of max 32587 tokens, 47.7 tok/s, 32 s
[strata] answering: 1543 of max 32587 tokens, 48.4 tok/s, 33 s
[strata] answering: 1610 of max 32587 tokens, 48.9 tok/s, 34 s
[strata] answering: 1670 of max 32587 tokens, 49.3 tok/s, 35 s
[strata] answering: 1733 of max 32587 tokens, 49.6 tok/s, 36 s
[strata] answering: 1793 of max 32587 tokens, 49.9 tok/s, 37 s
[strata] answering: 1850 of max 32587 tokens, 50.1 tok/s, 38 s
[strata] answering: 1909 of max 32587 tokens, 50.3 tok/s, 39 s
[strata] answering: 1971 of max 32587 tokens, 50.5 tok/s, 40 s
[strata] answering: 2037 of max 32587 tokens, 50.9 tok/s, 41 s
[strata] done: 2083 tokens in 42 s (51.1 tok/s) (stop, cancel=False), expert cache 72.5% hit

[VecSzn]

Thanks, that's a good result for an 8 GB card: 67.6 tok/s writing and 434 tok/s reading.

Setup doesn't know gfx1032 yet, so before adding it I need to know how you got it running:

1. Did you set HSA_OVERRIDE_GFX_VERSION=10.3.0, or build the engine for gfx1032 (gfx1032 in the build's GPU targets)? The command, or the change the AI made, is enough.
2. Which Qwen3.6 file (Q2_K_XL, Q3_K_M or Q4_K_M), and how much RAM does the PC have?
3. The log: when the server starts, the engine prints lines beginning with strata-poly: in the terminal you started it from, among them loaded in ... s; N of ... routed experts in VRAM. The first 10-20 lines after the start are all I need.

With that I can see what adding gfx1032 to setup takes, and then I'll ask you to try it.

[midnightx86]

Hi, i add it to setup.py:

------------------------------------------------------------------------------------------------ AMD

The RX 7900 XT / XTX (gfx1100) and the RX 9070 series / Radeon AI PRO R9700 (gfx1201) on Linux, through the HIP

backend (docs/AMD_HIP.md); the RX 7800 XT / 7700 XT (gfx1101, #254) and the RX 9060 XT (gfx1200, #256) were run by

their owners; the RX 6800 / 6900 series (gfx1030, #311) and the RX 6700 XT (gfx1031, #524) run but are unvalidated. There is no ready-made AMD engine: ROCm comes from AMD's TheRock Python wheels into .venv (no sudo;

a system ROCm 7 in /opt/rocm is used when it has hipcc and hipBLAS) and the engine is compiled here for the cards.

No images yet.

ROCM_INDEXES = {"gfx1100": "https://rocm.nightlies.amd.com/v2/gfx110X-dgpu/", # TheRock's wheels per GPU family
"gfx1101": "https://rocm.nightlies.amd.com/v2/gfx110X-dgpu/",
"gfx1102": "https://rocm.nightlies.amd.com/v2/gfx110X-dgpu/",
"gfx1200": "https://rocm.nightlies.amd.com/v2/gfx120X-all/",
"gfx1201": "https://rocm.nightlies.amd.com/v2/gfx120X-all/",
"gfx1030": "https://rocm.nightlies.amd.com/v2/gfx103X-all/",
"gfx1031": "https://rocm.nightlies.amd.com/v2/gfx103X-all/",
"gfx1032": "https://rocm.nightlies.amd.com/v2/gfx103X-all/",
"gfx1151": "https://rocm.nightlies.amd.com/v2/gfx1151/", # Strix Halo (docs/STRIX_HALO.md)
"gfx1103": "https://rocm.nightlies.amd.com/v2/gfx110X-all/"} # Radeon 780M: only with STRATA_EXPERIMENTAL_GFX1103=1

The TheRock nightly indexes are pruned and move on, and each GPU family's index holds its own range (#1103: gfx103X-all

starts at 7.13.0a20260422 and has no 7.10; #1267: the 7.10 wheel segfaults on a Strix Halo with kernel 7.2.8), so one

version for every card cannot work. The wheel is chosen per index (the family): a preferred version, tried first; when

the index no longer offers it, the newest one of the same 7.x line, else of the same major, with a warning (rocm_pick).

STRATA_ROCM_VERSION still forces one exact version (no lookup).

ROCM_VERSION_DEFAULT = "7.10.0a20251120" # what PolyStrata's HIP build was tested with (gfx120X, gfx110X)
ROCM_FAMILY_PINS = {"gfx103X-all": "7.13.0a20260515",
"gfx1032": "7.13.0a20260515", # #1103: 7.13.0a20260515 runs; the 7.14 nightlies time out
"gfx110X-all": "7.10.0a20251121", # the experimental gfx1103: this index has no ...20251120
"gfx1151": "7.14.0a20260608"} # #1267: 7.14.0a20260529 to 20260608 run on kernel 7.2.8; the 7.10 wheel segfaults
ROCM_VERSION_OVERRIDE = os.environ.get("STRATA_ROCM_VERSION") or None
ROCM_VERSION = ROCM_VERSION_OVERRIDE or ROCM_VERSION_DEFAULT # kept for callers that read one version
ROCM_SYSTEM_MIN = (7, 0) # an older system ROCm is passed over for the wheels (gfx1201 needs ROCm 6.4 or newer)

STRATA_EXPERIMENTAL_GFX1103=1 (opt-in, unsupported): the Radeon 780M / 760M / 740M iGPU (Ryzen 7040 / 8040, gfx1103) is taken

as a unified-memory AMD card like Strix Halo, with the portable kernels (no WMMA) (measured on one machine: Ryzen 7 255). Unset: unchanged.

GFX1103_OPT_IN = os.environ.get("STRATA_EXPERIMENTAL_GFX1103") == "1"
AMD_ARCHS = ("gfx1100", "gfx1101", "gfx1102", "gfx1200", "gfx1201", "gfx1030", "gfx1031", "gfx1032", "gfx1151") + (("gfx1103",) if GFX1103_OPT_IN else ())
AMD_NAMES = {"gfx1100": "AMD Radeon RX 7900 series (gfx1100)", # when sysfs has no product name
"gfx1101": "AMD Radeon RX 7800 XT / 7700 XT (gfx1101)",
"gfx1102": "AMD Radeon RX 7600 / 7600 XT (gfx1102)",
"gfx1200": "AMD Radeon RX 9060 series (gfx1200)",
"gfx1201": "AMD Radeon RX 9070 series / AI PRO R9700 (gfx1201)",
"gfx1030": "AMD Radeon RX 6800 / 6900 series (gfx1030)",
"gfx1031": "AMD Radeon RX 6700 XT series (gfx1031)",
"gfx1032": "AMD Radeon RX 6600 XT (gfx1032)",
"gfx1151": "AMD Radeon 8060S / 8050S / 8040S (Ryzen AI Max, Strix Halo, gfx1151)",
# integrated Radeons of other Ryzen families: named so they are not taken for Strix Halo (not supported)
"gfx1150": "AMD Radeon 890M / 880M (Ryzen AI 300, Strix Point, gfx1150)",
"gfx1152": "AMD Radeon 860M / 840M (Ryzen AI 300, Krackan Point, gfx1152)",
"gfx1103": "AMD Radeon 780M / 760M / 740M (Ryzen 7040 / 8040, Phoenix / Hawk Point, gfx1103)"}
AMD_CARDS = ("the RX 7900 XT / XTX (gfx1100), RX 7800 XT / 7700 XT (gfx1101), RX 9060 XT (gfx1200) and "
"RX 9070 / 9070 XT / Radeon AI PRO R9700 (gfx1201), and the RX 6800 / 6900 series (gfx1030) and RX 6700 XT "
"(gfx1031, #524), and the RX 7600 / 7600 XT (gfx1102, one run reported, #942), all unvalidated, and the Ryzen AI Max "Strix Halo" APU (Radeon 8060S / 8050S / 8040S, "
"gfx1151: experimental, docs/STRIX_HALO.md)")

How it was built and run:
We did both: we built specifically for gfx1032 AND used the environment variable override.

First, we edited the setup script code to add gfx1032 mapping (ROCM_INDEXES, ROCM_FAMILY_PINS, AMD_ARCHS, and AMD_NAMES) pointing to the gfx103X-all wheels with pin 7.13.0a20260515.

Then we built the engine targeting gfx1032:

CMAKE_HIP_ARCHITECTURES=gfx1032 ./setup.sh --backend hip --setup
And for running the engine, we set the runtime override:

export HSA_OVERRIDE_GFX_VERSION=10.3.2

I start the model with this run-qwen36-ud-q4_k_m.sh:
#!/bin/sh
cd "/home/MidNight/PolyStrata"
export HSA_OVERRIDE_GFX_VERSION=10.3.2
export STRATA_ALLOWED_HOSTS=host.containers.internal
exec "/home/MidNight/PolyStrata/.venv/bin/python" "/home/MidNight/PolyStrata/serve/server.py" "--engine" "strata" "--config" "/home/MidNight/PolyStrata/strata-qwen36-ud-q4_k_m.json" "--port" "8080" "--open" "--host" "0.0.0.0" "--api-key" "my-apy-key"

Im using the model Qwen3.6-35B-A3B (UD-Q4_K_M), i have 32 gb ram ddr5.

About the log, what im seeing now when i restart is this (maybe because is already cached):

(midnight_env) MidNight@fedora:~/PolyStrata$ ./run-qwen36-ud-q4_k_m.sh
loading the model (the first start takes a minute or two) ...
[strata] starting the engine: reading the model's weights ...
[strata] thinking level: none (reasoning_effort in the config; a request, or the Chat settings shared with other apps, can set its own)
[strata] also answers to the host names host.containers.internal (allowed_hosts)
ready: http://127.0.0.1:8080/v1 (OpenAI: /v1/chat/completions, Anthropic: /v1/messages, context 32768 tokens, API key required)
open http://127.0.0.1:8080/ in a browser to chat; close this window to stop the model
from other devices: http://192.168.1.171:8080/ (API: http://192.168.1.171:8080/v1)

Let me know if you need something else, thank you!

[VecSzn]

gfx1032 is in setup now (9c90afe on main): the RX 6600 / 6600 XT / 6650 XT take the same gfx103X-all wheels and pin as gfx1031, marked unvalidated, and your run is noted in docs/AMD_HIP.md.

A correction to my last message: the engine's own log isn't in the terminal. It's the file strata-qwen36-ud-q4_k_m.log in your PolyStrata folder.

To try it: put your own setup.py edits aside (git checkout -- setup.py if it's a git clone), update to main, then run ./setup.sh --backend hip --setup without CMAKE_HIP_ARCHITECTURES. Setup should find gfx1032 itself and build the engine for it. HSA_OVERRIDE_GFX_VERSION=10.3.2 is the card's own version, so an engine built for gfx1032 shouldn't need it; please try one start without it. If setup or the engine stops, the last 30 lines of that log are what I need.

[midnightx86]

Hi, i just tried, this is the log:

[123/124] Linking HIP executable strata-poly
[ok] strata-poly compiled: /home/MidNight/PolyStrata/engine-poly/strata-poly
[ok] engine: /home/MidNight/PolyStrata/engine-poly/strata-poly

=== Step 5: downloading Qwen3.6-35B-A3B UD-Q4_K_M ===
[ok] model file present

=== Step 6: preparing the model for PolyStrata ===
[ok] model prepared: /home/MidNight/PolyStrata-data/packs/qwen36-ud-q4_k_m

=== Step 7: writing the start script ===
Several requests at once (opt-in, --parallel N): on this card most experts run on the CPU and do each request's own, so requests at once gain less: on an 8 GB laptop three chats at once wrote about a third faster than one after the other (docs/BATCHING.md).
[ok] AMD card: --kv-whole, the attention caches take their VRAM at load (ROCm keeps what the expert cache gives back, so a long prompt was refused VRAM later)
[ok] kept from your earlier strata-qwen36-ud-q4_k_m.json: reasoning_effort, reasoning_close_retry
[ok] start script: run-qwen36-ud-q4_k_m.sh

All set.
API (OpenAI): http://127.0.0.1:8080/v1 (any API key; model name: anything)
API (Anthropic): http://127.0.0.1:8080/v1/messages
Next time: just run ./setup.sh (or run-qwen36-ud-q4_k_m.sh) - it starts right away

---

Starting qwen3.6-35b-a3b-ud-q4_k_m: it reads the model file (about 23 GB) in place: the weights go to the GPU,
and the OS file cache keeps the experts the GPU does not hold.
While it does, YOUR PC CAN BE SLOW OR STOP RESPONDING FOR 1-3 MINUTES (longer the first time after a
restart). That is normal: please wait and don't close this window - the browser opens when it is ready.
Later, closing this window stops the model.

Settings (strata-qwen36-ud-q4_k_m.json): --max-context 32768 --tail-skip 6 --kv-whole; server
127.0.0.1:8080, gpu 0
loading the model (the first start takes a minute or two) ...
[strata] starting the engine: reading the model's weights ...
[strata] still starting (20 s) - please wait ...
[strata] thinking level: none (reasoning_effort in the config; a request, or the Chat settings shared with other apps, can set its own)
ready: http://127.0.0.1:8080/v1 (OpenAI: /v1/chat/completions, Anthropic: /v1/messages, context 32768 tokens)
open http://127.0.0.1:8080/ in a browser to chat; close this window to stop the model
[strata] thinking limited to 23962 of max_tokens 31949 so the answer has room ("answer_room": false in the config turns this off)
[strata] the engine reported an error: rms_norm: device kernel image is invalid
[strata] done: 0 tokens in 0 s (0.0 tok/s) (error, cancel=False)

But when i write to the model: rms_norm: device kernel image is invalid

Using export HSA_OVERRIDE_GFX_VERSION=10.3.2 it doesnt happens the issue.

(midnight_env) MidNight@fedora:~/PolyStrata$ ./run-qwen36-ud-q4_k_m.sh
loading the model (the first start takes a minute or two) ...
[strata] starting the engine: reading the model's weights ...
[strata] thinking level: none (reasoning_effort in the config; a request, or the Chat settings shared with other apps, can set its own)
[strata] also answers to the host names host.containers.internal (allowed_hosts)
ready: http://127.0.0.1:8080/v1 (OpenAI: /v1/chat/completions, Anthropic: /v1/messages, context 32768 tokens, API key required)
open http://127.0.0.1:8080/ in a browser to chat; close this window to stop the model
from other devices: http://192.168.1.171:8080/ (API: http://192.168.1.171:8080/v1)
[strata] thinking limited to 23947 of max_tokens 31929 so the answer has room ("answer_room": false in the config turns this off)
[strata] reading the prompt: 415 of 831 tokens, 1 s so far
[strata] thinking: 63 of budget 23947 (max 31929) tokens, 67.5 tok/s, 2 s
[strata] thinking: 129 of budget 23947 (max 31929) tokens, 65.8 tok/s, 3 s
[strata] thinking: 205 of budget 23947 (max 31929) tokens, 69.0 tok/s, 4 s
[strata] thinking: 271 of budget 23947 (max 31929) tokens, 68.0 tok/s, 5 s
[strata] thinking: 349 of budget 23947 (max 31929) tokens, 70.0 tok/s, 6 s
[strata] thinking: 426 of budget 23947 (max 31929) tokens, 71.0 tok/s, 7 s
[strata] thinking: 493 of budget 23947 (max 31929) tokens, 70.4 tok/s, 8 s
[strata] thinking: 572 of budget 23947 (max 31929) tokens, 71.4 tok/s, 9 s
[strata] thinking: 660 of budget 23947 (max 31929) tokens, 73.2 tok/s, 10 s
[strata] thinking: 735 of budget 23947 (max 31929) tokens, 73.4 tok/s, 11 s
[strata] thinking: 815 of budget 23947 (max 31929) tokens, 73.9 tok/s, 12 s
[strata] thinking: 903 of budget 23947 (max 31929) tokens, 75.0 tok/s, 14 s
[strata] thinking: 977 of budget 23947 (max 31929) tokens, 74.8 tok/s, 15 s
[strata] thinking: 1059 of budget 23947 (max 31929) tokens, 75.2 tok/s, 16 s
[strata] thinking: 1136 of budget 23947 (max 31929) tokens, 75.3 tok/s, 17 s
[strata] thinking: 1211 of budget 23947 (max 31929) tokens, 75.2 tok/s, 18 s
[strata] thinking: 1279 of budget 23947 (max 31929) tokens, 74.7 tok/s, 19 s
[strata] thinking: 1358 of budget 23947 (max 31929) tokens, 74.8 tok/s, 20 s
[strata] thinking: 1423 of budget 23947 (max 31929) tokens, 74.2 tok/s, 21 s
[strata] thinking: 1490 of budget 23947 (max 31929) tokens, 73.8 tok/s, 22 s
[strata] thinking: 1568 of budget 23947 (max 31929) tokens, 74.0 tok/s, 23 s
[strata] thinking: 1638 of budget 23947 (max 31929) tokens, 73.8 tok/s, 24 s
[strata] thinking: 1713 of budget 23947 (max 31929) tokens, 73.7 tok/s, 25 s
[strata] thinking: 1788 of budget 23947 (max 31929) tokens, 73.7 tok/s, 26 s
[strata] thinking: 1859 of budget 23947 (max 31929) tokens, 73.6 tok/s, 27 s
[strata] answering: 1933 of max 31929 tokens, 73.5 tok/s, 28 s
[strata] answering: 2018 of max 31929 tokens, 73.9 tok/s, 29 s
[strata] answering: 2089 of max 31929 tokens, 73.7 tok/s, 30 s
[strata] answering: 2140 of max 31929 tokens, 73.0 tok/s, 31 s
[strata] answering: 2159 of max 31929 tokens, 71.0 tok/s, 32 s
[strata] answering: 2202 of max 31929 tokens, 70.1 tok/s, 33 s
[strata] answering: 2256 of max 31929 tokens, 69.5 tok/s, 34 s
[strata] answering: 2313 of max 31929 tokens, 69.1 tok/s, 35 s
[strata] done: 2331 tokens in 35 s (69.0 tok/s) (stop, cancel=False), expert cache 71.1% hit

[VecSzn]

The 69 tok/s on the RX 6600 XT is a good number for an 8GB card. The error "device kernel image is invalid" most likely came from HSA_OVERRIDE_GFX_VERSION being set somewhere else. The setup was only building the engine for gfx1032, but the RX 6600 XT is itself a gfx1032 card so it shouldn't need it. My guess is that your shell is already setting it to 10.3.0, a lot of RDNA2 guides say to put that in your .bashrc. 10.3.0 makes your card look like gfx1030, and the engine has no gfx1030 code. Try running echo $HSA_OVERRIDE_GFX_VERSION in a new terminal, not in the start script. If it prints 10.3.0, remove that line from your .bashrc or from midnight_env and it should start without the export.

[Nyno3333]

25 tok/s on my setup with an RX 6600 but a custom qwen 3.8, the unsloth 3.6 gives me even less somehow 20 tok/s and i cant use mtp for performence gain...😭

my pc have some stuttering issues in game/heavy-app, reinstalling my pilotes wasnt enough, i thinkwill reinstall windows next week to see how it goes after that.

[midnightx86]

Yes, it was that! thanks VecSzn! , now works without HSA_OVERRIDE_GFX_VERSION=10.3.2

Regarding the tokens, i already reach more than 90t/s with long coding task.
[strata] reading the prompt: 695 of 3,169 tokens, 1 s so far
[strata] reading the prompt: 1,468 of 3,169 tokens, 2 s so far
[strata] reading the prompt: 2,240 of 3,169 tokens, 3 s so far
[strata] reading the prompt: 3,013 of 3,169 tokens, 5 s so far
[strata] answering: 66 of max 4096 tokens, 79.9 tok/s, 6 s
[strata] answering: 158 of max 4096 tokens, 86.3 tok/s, 7 s
[strata] answering: 246 of max 4096 tokens, 86.8 tok/s, 8 s
[strata] answering: 343 of max 4096 tokens, 88.6 tok/s, 9 s
[strata] answering: 434 of max 4096 tokens, 88.9 tok/s, 10 s
[strata] answering: 520 of max 4096 tokens, 88.3 tok/s, 11 s
[strata] answering: 606 of max 4096 tokens, 87.4 tok/s, 12 s
[strata] answering: 697 of max 4096 tokens, 87.2 tok/s, 13 s
[strata] answering: 800 of max 4096 tokens, 88.6 tok/s, 14 s
[strata] answering: 907 of max 4096 tokens, 90.4 tok/s, 15 s
[strata] answering: 1001 of max 4096 tokens, 90.3 tok/s, 16 s
[strata] answering: 1077 of max 4096 tokens, 89.1 tok/s, 17 s
[strata] answering: 1178 of max 4096 tokens, 90.0 tok/s, 18 s
[strata] done: 1231 tokens in 19 s (89.5 tok/s) (stop, cancel=False), expert cache 60.0% hit
[strata] 400 invalid request: this server was started without the vision encoder (run setup again and choose 'vision'), so it cannot read images
[strata] reading the prompt: 105 of 4,314 tokens, 1 s so far
[strata] reading the prompt: 946 of 4,314 tokens, 2 s so far
[strata] reading the prompt: 1,788 of 4,314 tokens, 3 s so far
[strata] reading the prompt: 2,629 of 4,314 tokens, 5 s so far
[strata] reading the prompt: 3,471 of 4,314 tokens, 6 s so far
[strata] reading the prompt: 4,314 of 4,314 tokens, 7 s so far
[strata] answering: 92 of max 3000 tokens, 89.2 tok/s, 8 s
[strata] answering: 189 of max 3000 tokens, 91.2 tok/s, 9 s
[strata] answering: 290 of max 3000 tokens, 94.1 tok/s, 10 s
[strata] answering: 388 of max 3000 tokens, 94.8 tok/s, 11 s
[strata] answering: 480 of max 3000 tokens, 94.1 tok/s, 12 s
[strata] answering: 578 of max 3000 tokens, 94.7 tok/s, 13 s
[strata] answering: 672 of max 3000 tokens, 94.5 tok/s, 14 s
[strata] answering: 770 of max 3000 tokens, 94.8 tok/s, 15 s
[strata] answering: 869 of max 3000 tokens, 95.2 tok/s, 16 s
[strata] answering: 973 of max 3000 tokens, 95.6 tok/s, 17 s
[strata] answering: 1073 of max 3000 tokens, 96.0 tok/s, 18 s
[strata] answering: 1174 of max 3000 tokens, 96.2 tok/s, 19 s
[strata] done: 1223 tokens in 20 s (96.8 tok/s) (stop, cancel=False), expert cache 63.6% hit

[VecSzn]

Good, glad it works now without having to use the override. The 69 tok/s run and the override note have been added to docs/AMD_HIP.md.