fix(rocm): #785 runtime arch dispatch — launch SharedKWmma prefill where it exists - #1186
Merged
Merged
Conversation
Contributor
Author
|
Scope note: this PR enables the WMMA host launch d=256-only — the specialization the witness proves end-to-end ( |
added 5 commits
August 18, 2026 15:53
Host `#if VT_ROCWMMA_OK` deleted both launch sites. Gate on hipDeviceProp_t.gcnArchName prefix gfx1200/gfx1201, cached once per device. Device kernel body unchanged. Shipping f58b KD (gfx1201 object): - d=256 SharedKWmma<2,8,16,32,false>: vgpr=151 spill=0 private=0 LDS=4880 - d=512 SharedKWmma<2,16,16,16,true>: vgpr=192 spill=52 private=212 LDS=2576 P1 GPU HOLD. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Hermes:grok-4.6 [Hermes]
Researcher b1ba HOLD: shipping-f58b d=512 WMMA spills (52/212) and must not become default-on. Remove the d=512 host launch/stub. d=256 <2,8,16,32,false> stays. d=512 remains scalar SharedK. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Hermes:grok-4.6 [Hermes]
Test/runner-only. Product dispatch and kernel bytes unchanged vs 52ed5f8. Host fixture/oracle/trace classifier + fail-closed rocprofv3 runner. GPU remains HOLD. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Hermes:grok-4.6 [Hermes]
Shared fixture header, GPU hashes printed/checked, exclusive XOR classifier, add_executable-only GPU target, pinned A/B env. Product src/include unchanged vs 52ed5f8. GPU remains HOLD. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Hermes:grok-4.6 [Hermes]
Classifier now requires exact WMMA <2,8,16,32,false> or exact scalar <2,8,32,32>. Prefix family matches are UNKNOWN. Product src/include unchanged. GPU remains HOLD. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Hermes:grok-4.6 [Hermes]
localai-bot
force-pushed
the
fix/785-wmma-runtime-dispatch
branch
from
August 18, 2026 15:57
233c7dc to
0e61f07
Compare
localai-bot
pushed a commit
that referenced
this pull request
Aug 18, 2026
…e fourth time `origin/main` advanced twice more (#1245, #1186). Only `.agents/issue-index.md` overlaps, and it overlaps every time because it is the one file in this change that every other branch also appends to. That is why GitHub keeps reporting this pull request CONFLICTING: it does not run the `merge=union` driver `.gitattributes:7` sets, so the conflict is real on the forge and absent locally. Resolved the only way that keeps the file an append-only log: the auto-merge was DISCARDED, main's file taken whole, and only the rows whose KEY main lacks re-appended -- #1074 and #1080. Asserted: main's blob is a strict BYTE prefix of the result (346713 of 349709 bytes), 370 rows, zero duplicate keys. Nothing else overlapped. The delta against `ae581da3e` is the same five files it was against `10fe7f475`. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code]
localai-bot
added a commit
that referenced
this pull request
Aug 18, 2026
…tioning builder that has no frame-0 branch (#1096) (#1209) Closes #1096. Closes #1191. `pipeline_kind = keyframe_interpolation` resolves `KeyframeInterpolationPipeline` (`keyframe_interpolation.py:55` @ `fd4ded7f`) on all four generations the table keys. Asking for it used to get the generic table refusal. It was the last unported LTX-2.5 pipeline that is neither hardware- nor artifact-blocked. Structurally it is `ti2vid_two_stage`: the same parser, the same guided half-res stage 1 on the UNADAPTED model, the same Euler stepper (derived — neither `self.stage_1(...)` at `:231` nor `self.stage_2(...)` at `:271` passes one, so `utils/blocks.py:524-527` applies), the same frozen three-sigma stage 2, and the same `schedule_tokens = kSchedulerDefault` because `:200` calls `execute(steps=...)` with no latent. **Two fields differ, and both of them render either way.** ## The conditioning builder, which is what this pipeline is named after `:211` and `:260` call `image_conditionings_by_adding_guiding_latent` (`helpers.py:343-367`). Every other pipeline calls `combined_image_conditionings` (`:272-308`). The two differ by one branch: | | `combined_image_conditionings` | `..._by_adding_guiding_latent` | |---|---|---| | `frame_idx == 0` | `VideoConditionByLatentIndex` (`:295-300`) | `VideoConditionByKeyframeIndex` | | any other `frame_idx` | `VideoConditionByKeyframeIndex` (`:301-305`) | `VideoConditionByKeyframeIndex` | | what frame 0 does | REPLACES latent frame 0's clean tokens; the count never moves (`latent_cond.py:38-39`) | APPENDS a latent frame of tokens (`keyframe_cond.py:79-82`) | The second has no branch at all. The first image is a keyframe the model interpolates **from**, not a frame it overwrites. This engine hard-coded the other arm, and both conditioning primitives were already ported and gated — what was missing is the SELECTION, now `Ltx2PipelineRecipe::image_conditioning`, defaulting to today's behaviour so nothing landed moves, read in exactly one place. **Nothing about a render can see this.** The wrong builder returns a clip of the right size, the right frame count and the right sample rate with the image visibly present. It is conditioned; it is conditioned as a different pipeline. The only observable is the sequence length the DiT ran over, and the gate renders the same image at the same geometry on both kinds plus a **bare** control that pins the target grid — without which the comparison passes on a tree where both arms append. Measured: `video_tokens` 12 on `keyframe_interpolation` against 8 on `ti2vid_two_stage` and 8 bare, with `image_tokens` 4 on both. ## `audio_output_phase` is 1, and on `ti2vid_two_stage` it is 0 `:271` binds `video_state, audio_state = self.stage_2(...)` and `:293` decodes that name. `ti2vid_two_stages.py:289` binds `video_state, _` under its own comment at `:287-288`, "Stage 2 refines video only; discard its audio", and decodes the name `:247` bound. Neither file argues the point, so the binding is the statement — and copying the neighbour ships a soundtrack one refinement stage stale, at the right length and the right sample rate. The recipe case asserts both polarities beside `a2vid_two_stage`'s 1 and `res2s_two_stage`'s 0, so it cannot pass because every two-stage recipe happens to agree. ## Two of #1096's three blockers were stale, and the real difference was in none `.agents/issue-index.md:321` named a multi-keyframe request surface, a per-sigma guided denoiser, and two missing checkpoints. All three were re-derived at the pin rather than inherited: - **The per-sigma denoiser is stale on this pipeline's default path.** `main()` passes plain `MultiModalGuiderParams` (`:325-340`), never a factory, so `create_multimodal_guider_factory` takes its last line, `MultiModalGuiderFactory.constant` (`guiders.py:360`), which builds `_params_by_sigma = ((inf, params),)` (`:312-315`) — ONE bin, so `build_from_sigma` returns the same guider at every sigma and `FactoryGuidedDenoiser` delegates to `_guided_denoise`, which is `Ltx2GuidedDenoise` (landed `daeff67f2`). The sigma-BINNED arm, `MultiModalGuiderFactory.from_dict`, is real, is reachable only by a caller who constructs a factory, and is now #1187. - **Both checkpoints are on the NAS**, byte-verified, and #1148 closed the pure-BF16 DiT refusal at `40a796aa9`. What is owed is the RUN. - **The multi-keyframe surface is true and is not the blocker.** Two pinned keyframes at the two ends is what interpolation means at its default configuration. An interior `frame_idx` is #1187. ## `--last-frame` on `ltx2-gen` (#1191, in flow) `vllm_video_params` has carried `last_frame` and the engine has served it since #930; `ltx2-gen` parsed `--first-frame` and never read the field. That only starts to bite on a pipeline whose whole job is the motion between two pinned frames, so this row is the first caller it narrows. One flag, parsed and assigned beside `--first-frame`, sharing the `--image-crf` and strength the two slots already share. ## The gate Whole binaries, never a `--test-case` filter. Focused, at this head: | Binary | cases | assertions | exit | |---|---|---|---| | `test_ltx2_pipeline` | 56 | 3316 | 0 | | `test_ltx2_video` | 87 | 2713 | 0 | RED before the recipe landed, on the same binaries: `test_ltx2_pipeline` 56 / 54 passed / 3183 assertions, `Status: FAILURE!`, exit 1, both new cases throwing `Unsupported LTX pipeline kind/version: 'keyframe_interpolation'/'2.5'`; `test_ltx2_video` 87 / 83 passed / 2593, exit 1. Full gate at `251f76b5e`, after merging `origin/main`: `CONFIGURE_EXIT=0`, `BUILD_EXIT=0`, `: error:` count 0, `ctest -N` **515**, `CTEST_EXIT=0`, `100% tests passed, 0 tests failed out of 515`. `No space left` and `BFD` are each 0 in the build and ctest logs, against injected controls that returned 1. Load average 50 to 76 across the run and 24 GiB free; none of the four load-dependent gates (#618, #294, #1052, #428) went red at that load. The same gate ran green at `1d880bcc5` before the merge, at 511 registered tests. The `READER ANCHORS` instrument in `ltx2_video.cpp` was ARMED rather than assumed correct: inserting one line above `kKnownLoadExtras` shifted every derived anchor by one and the gate went RED naming both lists. On the real tree recorded and derived agree at `[823 833 834 896 992 1008 1056 1147 1172 1277 1318 1360 1362]`, because this row's edits are below the last anchor. The tree was restored byte-for-byte after every mutation, with `os.utime` and a confirmed ninja recompile each time; `git status` is clean at the pushed head. ### Nine mutations, each printing four facts `git diff --stat`, BUILT, the `: error:` count, and the exit code captured directly — with the expected OLD line content asserted unique before every edit. | # | mutation | diff | built | errors | result | |---|---|---|---|---|---| | M1 | `recipe.image_conditioning` assignment deleted | 1 deletion | YES | 0 | video rc=1 (1 case), pipeline rc=1 (2 cases) | | M2 | reader forced to the REPLACE arm | 1+/2- | YES | 0 | video rc=1 (1 case) | | M3 | `audio_output_phase` 1 -> 0 | 1+/1- | YES | 0 | pipeline rc=1 (2 cases) | | M4 | `stage1.schedule_tokens` deleted | 1 deletion | YES | 0 | video rc=1, pipeline rc=1 | | M5 | `stage1.loras = kNoAdapters` deleted | 1 deletion | YES | 0 | pipeline rc=1 | | M6 | `requires_distilled_lora` deleted | 1 deletion | YES | 0 | video rc=1, pipeline rc=1 | | M7 | the dispatch key made UNREACHABLE | 1+/1- | YES | 0 | **all six new cases RED**, by name | | M8 | the COND arm left in velocity space | 3+/1- | YES | 0 | video rc=1, 7 cases / 13 assertions | | M9 | the anchor case's `gen.steps` 3 -> 2 | 1+/1- | YES | 0 | video rc=1 (2 assertions) | **M7 was run twice and the first run is recorded rather than dropped.** Deleting the whole dispatch arm left `KeyframeInterpolationRecipe` unreferenced and `-Werror` killed the build — `BUILT=NO`, 1 error, no test result at all, which is exactly the shape that reads as a passing test. Renaming the dispatch key keeps every line compiled and referenced while making the recipe unselectable through the request surface, which is what the reachability mutation is there to measure. M9 is the mutation `ltx25-ti2vid-recipe.md`'s first head passed. The step count comes back OUT of the render lambda and the trajectory recomputation runs at it, with `rendered_steps > 2` asserted by name, because at two steps `stretch` pins both non-zero sigmas and the schedule is `{1, 0.1, 0}` for every token count. Measured here: `keyframe: 4096 / 4096 res2s: 2 / 8`. ## What is NOT verified **No real-weights render.** Upstream marks this arm `Full + distilled LoRA` (`packages/ltx-pipelines/CLAUDE.md:24`), so stage 1's identity is CFG on the UNADAPTED model and the checkpoint it needs is `ltx-2.5-22b-dev-transformer-bf16.safetensors`. It is on the NAS and loadable. What is owed is a GPU lease and the two renders; another agent holds `dgx:gpu0` and no GPU work is in this row's scope. Running the arm against a **distilled** checkpoint instead would be worse than not running it — the distilled scales are trained into those weights, so a CFG-guided stage 1 samples a trajectory they were never trained for and renders a plausible clip with no diagnostic (#1137). The reach claim rests on the ABI path — `LoadVideoEngine` + `Generate`, which is what `ltx2-gen` drives. #928 does not exclude the HTTP route here, because all three knobs are LOAD extras and `requires_audio_input` is false; that is a claim about the request surface, and no case here drives HTTP end to end. ## Review repair A fresh review of `251f76b5e` returned eight findings. Six are repaired in `0e4357707`; the other two are recorded rather than silently absorbed. The blocking one was a bug this row introduced two hundred lines from the code that carries it. The last-frame arm located its own appended tokens at `positions[target_tokens * 2]`, the first token past the fixed target grid -- which is its own token only while this arm owns the first append. This row put a second appending item in front of it, so with both ends pinned the index named the FIRST frame's keyframe at temporal 0 and the arm threw. That is `docs/USAGE.md`'s worked example for this kind and what `ltx2-gen --help` tells the reader to do, so the documented headline command did not run. Neither assertion is weakened. Each arm now captures the sequence length at the moment of its own append and locates its tokens from that, so no arm depends on being first. The generated-keyframe-slot arm carried the same derivation and is repaired the same way. Three guarantees the review found ungated are addressed, and one of the three turned out to be a different problem than reported. `frame_idx = 0` and the trace's tail slice are now gated. `causal_fix = true` at that call site is INERT rather than merely ungated, measured on a probe: at `num_pixel_frames = 1`, which both production arms pass, flipping it moves 0 of 48 position values, because the temporal start clamps to 0 either way and the `num_pixel_frames == 1` narrow overwrites the end the fix moved. No call-site check can detect that flip. The risk lives in the `frame_idx == 0` gate the argument passes through, and that is gated in `test_ltx2_vae` at `num_pixel_frames != 1`, where it shows. Two records are corrected: `ti2vid_two_stages.py:211` is blank and the real `combined_image_conditionings` calls are `:231` and `:276`; and the claim that this row moved #1150's owed count from six to five was wrong on both numbers and on the premise, since the keyframe arm was unported rather than divergent. Both of those corrections stand. Closes #1219. #1220 is filed and NOT fixed: the two schedule-anchor cases return the request step count where their comment claims to read the render. It still catches the mutation it was built for, so it is a weakened guard rather than a vacuous one, and re-deriving it changes what a landed case measures on both pipelines. Listed under the row spec's `## Owed`. ## The scoped re-review: two off-by-N anchors, deferred whole A scoped re-review of `87e9f0e37` returned two findings with one root cause. An earlier shape of the repair corrected two inherited off-by-N upstream anchors on the seven new lines that restated them. Those corrections are now REVERTED and the whole question is deferred to [#1230](#1230). **Both readings are right, and that is not the point.** Re-derived here at the LTX-2 pin `fd4ded7f` by reading the pinned files rather than inheriting the citation: `latent_cond.py:38` is `latent_state = latent_state.clone()` and `:39` is blank, so the two writes are `:40-41`; `schedulers.py:31` is the return annotation `) -> torch.FloatTensor:`, so the `tokens = math.prod(latent.shape[2:])` read is `:32`. Correcting seven of the twenty-two citations was still wrong, because **a partial correction is strictly worse than none.** A uniformly wrong anchor is one grep from being right, and a whole tree citing `:38-39` is a single mechanical edit for a single reviewer. A file citing BOTH forms is not. `src/vllm/multimodal/ltx2_video.cpp` read `latent_cond.py:38-39` at `:2208` and `:3174` and `latent_cond.py:40-41` a hundred lines later at `:3392`, with nothing in the tree recording which one to believe -- so the one file a reader of this change opens was the one file stating both. The same split ran through `schedulers.py:31` in `tests/vllm/multimodal/test_ltx2_video.cpp`, whose two near-identical schedule-anchor comments at `:7890` and `:8485` disagreed after the correction and agree again now. It also **cost a gate, which is the blocking half.** `include/vllm/model_executor/models/ltx2_pipeline.h:754` was one of the seven, and `USER_USAGE_PREFIXES` in `scripts/check-doc-checkpoint.py:99` is a pure path match on `include/vllm/` with no content analysis. A one-line comment edit in a public header therefore reads as a usage change and demands a `docs/USAGE.md` edit. `.github/workflows/ci.yml:448` runs that gate per commit over the range, so no follow-up commit could clear it -- the commit itself had to change. There is no user-visible usage change here, so writing a `docs/USAGE.md` edit to turn the gate green is the move AGENTS.md forbids. Removing the header edit is the honest fix, and it retires the mixed file in the same stroke. **The repair commit no longer touches `include/` at all.** Measured, on the rewritten repair commit `263f82b67`: | range | `check-doc-checkpoint.py` | |---|---| | `--base 5af6e76 --head 87e9f0e` | exit **1**, `commit 0e43577: changed user_usage but did not update docs/USAGE.md` | | `--base 5af6e76 --head 853384a` | exit **0**, `OK: public documents match the claims this change makes` | The old head is kept as the control so the instrument is shown armed rather than assumed: the checker still reds on the pre-rewrite range from the same working tree that returns 0 on the new one. It is also 0 against today's `origin/main`. **The RECORDS go the other way on purpose and are not reverted.** The spec's port-map table and #1219's index row state `:40-41` and `:32`, because a record's job is to say what is true, and the index is append-only, so a wrong anchor written there could never be swept. What the record now says, and what #1230 carries, is: the anchors are `:40-41` and `:32`, the source cites them uniformly short in twenty-two places, and one row corrects all of them at once. The tree already carried eight citations of the CORRECT form before this row existed -- `include/vllm/multimodal/ltx2_video.h:625` among them -- so the mixture is older than this change and outlives it either way. #1230 is filed, listed under `## Owed` in the row spec and appended to the issue index, and stays OPEN. ## Gate rerun by the operator Rerun at `87e9f0e37`, not taken from the implementer's report. `CONFIGURE_EXIT=0`, `BUILD_EXIT=0`, `: error:` 0, no `No space left`. `ctest -N` = **516** and `ctest -j3` = **100% tests passed, 0 tests failed out of 516**, `CTEST_EXIT=0`. Focused, with ANSI stripped and `Status:` read rather than inferred from the exit code: `test_ltx2_pipeline` 56 cases / 3316 assertions, `test_ltx2_video` **88 / 2755**, `test_ltx2_vae` 43 / 3125, all `Status: SUCCESS!` at exit 0. The video case count moved from the reviewed head's 87 / 2713, which is what proves the new both-keyframes case ran rather than matching nothing -- a doctest filter that selects zero cases prints `SUCCESS!` and exits 0. `windows-msvc-cpu` and `windows-msvc-vulkan` are red. They are red on every PR in this repo -- confirmed on the unrelated #1186 and #1178 -- because those jobs are PR-only with no `main` baseline. Not attributable to this change. ## Gate rerun after the anchor revert Rerun by the repairing session at `853384a8b`, on the merged tree, in its own worktree. `CONFIGURE_EXIT=0`, `BUILD_EXIT=0`, `: error:` count **0**, `No space left` and `BFD` each **0**. `ctest -N` = **516** and `ctest -j3` = **100% tests passed, 0 tests failed out of 516**, `CTEST_EXIT=0`. Focused, whole binaries with no `--test-case` filter, ANSI stripped and `Status:` read rather than inferred from the exit code: | Binary | cases | assertions | `Status:` | exit | |---|---|---|---|---| | `test_ltx2_pipeline` | 56 | 3316 | `SUCCESS!` | 0 | | `test_ltx2_video` | 88 | 2755 | `SUCCESS!` | 0 | | `test_ltx2_vae` | 43 | 3125 | `SUCCESS!` | 0 | Unchanged from the operator's rerun at `87e9f0e37`, which is the expected result: the revert touches eight comment lines and no executable statement. `test_ltx2_video` holding at 88 rather than falling back to 87 is the load-bearing one, because a lost case is how a history rewrite drops work silently. Record gates on the rewritten range: `check-commit-trailers` OK, `check-commit-style` OK, `check-issue-index-append-only` OK, `check-now-current` OK, `check-agent-record` OK. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [claude-code] --------- Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
localai-bot
pushed a commit
that referenced
this pull request
Aug 18, 2026
…or it still lacks (#676) record(GEMMA4-ROCM-KEEP): the 2x R9700 KEEP recipe, and the denominator it still lacks Records the contributor KEEP recipe for Gemma-4-26B-A4B-it FP8 on dual R9700 (gfx1201, ROCm 7.2.4) and what it is still missing. Documents only: no kernel change, no default change, no env change. ## What is recorded Fair protocol `PREFIX_CACHE=0` plus unique pads, 2026-08-13. | Depth | median prefill t/s | |------:|-------------------:| | ~3k | 2112 | | ~11k | 2014 | | ~18k | 1705 | | ~42k | 1099 | Decode stream 55.5 t/s at temp=0, 49.1 t/s at temp=0.7. Paris, arith `63` and `gemma4` `tool_calls` held. Same-box Vulkan Q8 unique-pad prefill is 3503 @11k and 2714 @42k, so the ROCm path is 1.74x and 2.47x behind it there. The rejected levers are named with their numbers: FMHA_WMMA2 (quality fail plus 0.67x @11k), isolated P1 cm1 wg256 (~1.13x where ~3.35x isolated is needed), layer-split FIFO (~0.60x), Head-TP peer-read, hipBLASLt dual-GPU. So is the residual: the faithful HIP cm1 hsaco spills 339 VGPR against RADV/ACO's 0 on an equivalent llama.cpp coopmat1 spec, which is the mechanism behind the isolated 1.13x and is not yet a named LLVM component. ## What is NOT recorded, and why **There is no denominator.** Every number above is engine-side with nothing on the other side of it — no pinned vLLM-ROCm run on the same box, model, quantization, request shape, concurrency and cache policy. Under AGENTS.md "Gates" that is not a throughput result, so `docs/BENCHMARKS.md` keeps this backend at `PENDING: no binding throughput number` and this PR does not touch that row. `.agents/benchmark-record.md` is also deliberately not appended: it is the append-only measurement log, and a figure that enters it gets quoted afterwards as measured. The oracle is not hypothetical. `docs/ROCM.md` §5 documents two working Docker vLLM-ROCm recipes on this hardware family, the second building this project's pinned commit `555967922` inside `rocm/vllm-dev:base` in about 6.5 minutes. What is missing is a run, and only the contributor has 2x R9700. **The recipe is not reproducible as written.** Four of the names in it are read by no product code in this tree. `VT_ATTN_DECODE_KV_SPLITS`, `VT_ATTN_DECODE_SLIDE_SPLITS` and `VT_ATTN_DECODE_SPLIT_WARPS` occur only in `tests/vt/test_gemma4_rocm_fp8_seams.cpp`; `VT_ATTN_DECODE_SLIDE_WARPS` occurs nowhere at all; `git grep` over `src/` and `include/` returns zero hits for all four. That is #845, whose seam test asserts `EnvInt(name, 16) == 16` with the variable unset — a tautology that passes whether or not the knob exists, which is how names nothing reads came to look real. The `**Status:**` line names the run's tree as `PR tip feat/gemma4-rocm-fp8-split` with no commit SHA, so this repository cannot establish which tree produced the numbers. Either the four decode splits were live on that tree and it is not this one, or they were inert and the shipped defaults produced these figures. **Only the contributor can say which**, and until he does the decode figure has no recipe behind it. Both items are listed under `## Owed` in `.agents/specs/gemma4-rocm-fp8-moe.md`. ## Repairs in this revision `docs/ENVIRONMENT.md` described `VT_GEMMA4_PREFILL_GEMM_M` as default `256` over `16..2048`. `gemma4_moe.cpp:1016-1021` accepts `16..8192` and returns `2048`, and has since 2026-08-10. The row was already wrong on `main` and this branch edits that exact row, so it is repaired in flow. The same edit had dropped a recorded measurement — lab `512` ~+37% prefill vs `64` — and AGENTS.md says to move evidence, never to drop it, so it is restored beside the `512`→`2048` ~+80 eng @11k result that explains the default. `PEER_ACT` and `PREFILL_GEMM_M` are now spelled as the product spells them, `VT_GEMMA4_PREFILL_PEER_ACT` and `VT_GEMMA4_PREFILL_GEMM_M`, and both are already the default. `docs/USAGE.md` publishes only knobs a reader can actually set and says outright that the decode figure is not reproducible from them; the spec keeps the full as-run recipe, because a record holds what happened and a user page has to be followable. `docs/FEATURES.md` returns to `main`'s text. A throughput figure's home is `docs/BENCHMARKS.md` under the projection table, this change alters no feature, backend or quantization surface, and fitting the number into a cell already 219 of its 220 characters cost the row its `VT_GEMMA4_*`/`VT_ATTN_*` pointer and the `test_gemma4_rocm_fp8_seams` seam name. The title no longer says "plateau". The PR body always said the right thing — that this does not close the Vulkan bar, and it names the ACO disparity as the next hypothesis — but the title is what survives into `git log`, and "plateau" reads there as the ceiling claim AGENTS.md forbids. The branch was rebuilt by rebase onto `affc2a7fd`, so it carries no untrailered merge commit and both original commits keep their authorship. The `docs/USAGE.md` conflict against #837's landed GetBlas text was resolved as a union. ## Known-unrelated CI `windows-msvc-cpu` and `windows-msvc-vulkan` are red on every open PR from a break predating this branch (#503, #584). The previous `sanitize-cpu (address,undefined)` red on this branch was `test_ltx2_video`, from a lane 215 commits ahead of the old base; the rebase carries it away. ## Landing note (maintainer) Merged as a record-only change. The `CHANGES_REQUESTED` review it carried asked for a `docs/BENCHMARKS.md` evidence anchor or for the numbers to be marked non-binding; this branch takes the second option explicitly, leaves `docs/BENCHMARKS.md` untouched at `PENDING: no binding throughput number`, and names the four decode knobs that are read by nothing outside `tests/vt/test_gemma4_rocm_fp8_seams.cpp`. That review predates the rework and is stale against this head. One statement here is now dated in the contributor's favour rather than against it. The record says the prefill figures do not reproduce from `main` because of #785. That fix has since landed as `ae581da3e` (#1186), so the prefill half of the recipe is reachable from `main` as of this merge. The decode figures remain lab-only and non-reproducible, which is the part this record exists to say. `windows-msvc-cpu` and `windows-msvc-vulkan` are red as they are on every pull request in this repository and carry no verdict here; the other 18 gates passed. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: ClaudeCode:claude-opus-5 [ClaudeCode]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #785 (taken over from @joral with his blessing).
Problem
VT_ROCWMMA_OKis defined only under__gfx1200__/__gfx1201__, which exist only on the device compilation pass — the host-side#if defined(VT_ROCWMMA_OK)guard around bothPagedAttnPrefillSharedKWmmalaunches is therefore always false, and the launches are compiled out of the host TU entirely. The rocWMMA prefill path at d=256/d=512 has never executed; every such prefill silently falls through to the scalar kernel.Fix
Host-side runtime arch dispatch: query
hipDeviceProp_t::gcnArchNameonce per device (prefix matchgfx1200/gfx1201, handles suffixed forms likegfx1201:xnack-), launch the WMMA kernel on match, existing scalar path otherwise. Device-side kernel-body guard unchanged, so non-gfx120x fatbin slices still never contain (or launch) WMMA stubs. Decision cached per device — not per build, not per call.Evidence (2× R9700, gfx1201, shipping ROCm 7.2.4)
Kernel-trace witness on the product seam, A/B on the same build:
PagedAttnPrefillSharedKWmma<2,8,16,32,false>dispatch, no scalar family — the kernel launches for the first time from an unhacked treePagedAttnPrefillSharedK<2,8,32,32>dispatch, no WMMA familyf58b06d): 0 spill/private, wave32Expected impact
Our lab (which hard-enabled this path locally) measures the KEEP recipe at ~2,014 tok/s @11k / ~1,099 @42k prefill on 2×R9700 — numbers that were previously unreachable from
mainbecause the kernel never ran (see the reproducibility discussion on #676). This PR makes them reachable with the published knobs.