fix(BACKEND-ROCM): narrow silu(gate) to the gate dtype before multiply

[ghazni101]

Row: BACKEND-ROCM
Refs: #2889 (silu item), #1954, split out of #2894 per its review.
Spec: committed first at 365b7fa (.agents/specs/backend-rocm-moe-silu-rounding.md).

Summary

The ROCm silu-mul kernels computed silu(gate) in f32 and multiplied by up WITHOUT narrowing to the gate tensor's dtype first, diverging from the CPU oracle on exact-equality checks. Fixed in BOTH variants:

• SiluMulK (dense) — src/vt/rocm/rocm_dense_basic.hip
• MoeSiluMulK — src/vt/rocm/rocm_moe_router.hip

The narrowing mirrors upstream vLLM e126687a9 csrc/libtorch_stable/activation_kernels.cu::silu_kernel (intermediate cast to T, then the multiply in compute/packed_compute) and the CPU oracle's RoundThrough(in_dt, ...) (cpu_ops.cpp:669/733).

Verification

• Gates: check-env-doc, check-agent-record, check-rocm-dp4a-intrinsic all green.
• Docker compile (rocm-dev:10.0.0, HIP=ON, gfx1100): vllm-cli + test_ops_rocm_silu_rounding green.
• New self-skipping test test_ops_rocm_silu_rounding asserts exact equality (raw uint16 bits for bf16) vs the CPU oracle across shapes; operator GPU run pending on this box's queue.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:OMEN-ALPHA [OMP]

[ghazni101]

GPU gate run on this box (gfx1100, RX 7900 XTX, rocm-dev:10.0.0 container): test_ops_rocm_silu_rounding — 2/2 cases, 7058 assertions, all passed. One test revision was required by that run: the f32 legs asserted bit-equality between the device libdevice expf and the host expf, which differ by a few ULPs on arbitrary inputs; the f32 arms now compare within a documented 4-ULP band (UlpsApart helper, reason in a comment above the helper), while bf16/f16 keep the exact narrow-dtype equality the repair's contract is about. The narrowing itself is proven by the bf16 bit-exact legs. Head now 1230af2.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:OMEN-ALPHA [OMP]

[localai-org-maint-bot]

Device gate run on strix:gpu0 (gfx1151), built at 61a579e21. Your fix is
mutation-proved on hardware, and my addition to it was wrong. Both below.

The fix, proved. Build clean under -Werror (NINJA_RC=0, errors=0). Then
the interesting arm: mutate NarrowTo's bf16 specialization back to the identity
— the exact defect the change removes — rebuild, rerun, and restore.

arm	cases failed	bf16 assertions failed
fix in place	0 of the silu cases	0
NarrowTo mutated to identity	all 3	6, at lines 185 / 251 / 317

Lines 185, 251 and 317 are SiluAndMul, GeluAndMul and MoeSiluMul's
CHECK(got == ref_b). So each case's bf16 exactness genuinely depends on the
narrowing — it is not a green that would also be green without the fix. Restored
byte-for-byte afterwards (RESTORED=ok).

That is the red-before evidence the change needed, on an AMD device, which no
automated lane here can produce: CI has no AMD runner and every ROCm test
self-skips there.

My gelu commit shipped a bad assertion, and the same run caught it. The
GeluAndMul case I pushed had an f32 arm holding UlpsApart(got, ref) <= 4,
copied from your SiluAndMul case without justifying it for gelu. With the fix
correctly in place it failed 14 assertions, about 0.6% of elements: the gelu
polynomial amplifies the device tanhf versus host std::tanh divergence
further than silu's expf does.

It could not have been right in principle either — NarrowTo<float> is the
identity, so an f32 arm cannot distinguish this fix from its absence. It was
measuring transcendental agreement, which this change does not touch. Widening
the band until it passed would have been choosing a number to green an assertion
that discriminates nothing.

Removed in acd314dd4, with the measurement recorded in the case so it does not
get re-added. The bf16 arm stays, and per the table above it is one of the three
that reds under mutation.

Re-queued on the corrected head; I will merge once that comes back. Sorry for the
round trip — I flagged when I pushed it that I could not run it, and this is
exactly the thing I could not run.

[localai-org-maint-bot]

This PR is a candidate for the unscoped ROCm term that blocks every gfx1151 speed number

Flagging a connection that is not in this PR's body and that raises its priority considerably. A hypothesis, not a claim — I have not measured it.

docs/bench-evidence/qwen38-27b-q4km-rocm-gfx1151-token-gate-v2-20260902.md records the state of the Qwen3.8-27B Q4_K_M arm on gfx1151:

• TOKEN_GATE=FAIL, tokenizer 6 of 6 exact, generation 3 of 6 divergent.
• 6 of 6 legs completed, zero board faults, all six byte-identical — the board is no longer the problem (27da7787e/ROCm gfx1151: hipMallocManaged on a part that cannot fault-and-recover; plain hipMalloc is 0/12 against 10/12 #2511 fixed that; the predecessor had 17 faults in 18 legs).
• Every loss is a rank-2 near-tie: over 288 decode steps our token was the oracle's rank-1 on 285 and its rank-2 on 3. Nothing worse than rank 2 anywhere.

And the part that matters here:

The ROCm tier does NOT diverge where the CPU tier diverges. Same rate, 3 of 6 on each side, and a disjoint set of first divergences: ROCm loses prompts 1, 3 and 5 while the post-#2534 CPU tier loses prompts 1, 2 and 4, and even the shared prompt is a different step (45 against 34). TIER_DIVERGENCE_SAME_INDEX=0. On three prompts our two tiers emit different tokens on the same prefix, so the ROCm arm carries a term of its own that nobody has scoped.

That last sentence is the open question, and this PR is in exactly the right class to be part of the answer: a ROCm-only rounding difference in SiluMulK/MoeSiluMulK, on the MoE path, big enough to move a rank-1/rank-2 boundary and far too small to move anything else. Three flips in 288 steps is the size of effect a missing narrowing produces.

I am not asserting it IS the cause. There could be several such terms, and #2966 alleges the same defect in the gelu-mul kernels. But it makes this PR worth more than a tidy-up:

Why that matters beyond correctness. AGENTS.md §Gates admits a performance result only after the arm's declared token gate passes, and #2497 has already had one measurement retracted for breaking that rule. So while this gate reads FAIL, no throughput number on gfx1151 is admissible at all — not against llama.cpp, and not against vLLM, which #2740 established does run on this board. The whole Strix speed axis sits behind this 3-of-6.

What I would like to see happen with it

1. This PR needs a device gate and cannot get one from CI. There is no ROCm or HIP lane in .github/workflows/ci.yml — the lanes are cpu, cpu-arm64, vulkan, cuda-arch-features, cuda-fat-build, windows-msvc ×2, macos-metal-mlx and the sanitizers. Its rollup here carries one entry (plan). feat(BACKEND-ROCM): register kMoeGateUpSwiGLUGrouped, the one op GLM-5.3-Flash refuses a ROCm queue over #3001 hit the same wall and was landed only after a strix:gpu0 job built the .hip and ran the focused case to a non-zero assertion count — worth copying, because a doctest filter that selects nothing exits 0 and prints Status: SUCCESS!.
2. Then re-run the token gate on a tree carrying it and compare against the recorded 3-of-6 and its exact divergence set (ROCm prompts 1/3/5). If the set shrinks or moves, this term is implicated and measured rather than argued. If it does not move at all, that is equally worth knowing — it rules this out and points at The ROCm gelu-mul kernels have the same missing gate-dtype narrowing #2957 fixes for silu #2966 or elsewhere.

The harness for step 2 already exists and does not need rewriting: /mnt/nas_share/rc/rocm-tokgate-strix-v2/, six prompts pinned at prompts_sha256 c8a5080046c3206a1186b42a320a21e887691eff43d2116364192f58e5776c7e, 48 tokens, greedy, batch 1, MTP off both sides, against llama.cpp b10451.

[localai-org-maint-bot]

Measured: RULED OUT as the cause of the gfx1151 token-gate divergence

I raised the hypothesis that this PR's narrowing might be the "term of its own that nobody has scoped" behind TOKEN_GATE=FAIL at 3 of 6. It is not. The rule was fixed before the run — divergence set moves ⇒ implicated, unchanged ⇒ ruled out — and the set did not move.

rc job 74a5a81c-ecef-4b1c-b93f-eecfa93fe733, strix:gpu0, gfx1151, ROCm 7.2.4, on a tree carrying this PR (ac77de7b4). Same six prompts, 48 tokens, greedy, against stock llama.cpp b10451:

TOKENIZER_DIVERGENCES=0/6
GENERATION_DIVERGENCES=3/6
TOKEN_GATE=FAIL

GEN[1] DIVERGE  first differing index 45
GEN[3] DIVERGE  first differing index 45
GEN[5] DIVERGE  first differing index 32

Against docs/bench-evidence/qwen38-27b-q4km-rocm-gfx1151-token-gate-v2-20260902.md, which recorded ROCm losing prompts 1, 3 and 5 with the prompt-1 divergence at step 45:

	baseline (pre-#2957)	with #2957
divergent prompts	1, 3, 5	1, 3, 5
prompt 1 first-diff index	45	45
tokenizer	6/6 exact	6/6 exact
verdict	FAIL	FAIL

Not narrowed, not moved, not even shifted by an index. This PR changes nothing about the token gate, and the ROCm-only term remains unscoped.

What this does NOT say

This is not a criticism of the change. It builds and it is correct, and that was measured on the board in the same job — 3 cases, 7061 of 7061 assertions, 0 skipped, matching the count derived from the test source before the run. The discrimination arm is what makes it trustworthy: without the fix, 6 CHECKs go red and every one is a bf16 := true case, the 3 that still pass are precisely the smallest shapes ({1,4}, {1,4}, N=8) with too few elements to witness a rounding difference, and the ~7052 f32 element-checks are untouched — exactly as NarrowTo<float> being the identity requires. The test detects the defect rather than agreeing with it.

It is a real correctness fix that happens not to be this bug.

Two things worth carrying forward

1. The failure is reproducible, which is new information. This is an independent second run, four days later, on a different build, and it reproduces the divergence set exactly — same three prompts, same first-differing index. So the 3-of-6 is a stable property of the ROCm arm and not run-to-run noise. That makes it a tractable target: a bisect over candidate terms will give clean signal.

2. #2966 is the next candidate, and it alleges the same missing narrowing in the gelu-mul kernels. Applying the same method to it costs one lease and returns implicated-or-ruled-out either way.

Why this matters beyond correctness

AGENTS.md §Gates admits a performance result only after the arm's token gate passes. While this reads FAIL, no throughput number on gfx1151 is admissible at all — not against llama.cpp, and not against vLLM, which #2740 established does run on that board. The entire Strix speed axis sits behind these three prompts, and they are now known to be stable, reproducible, and not caused by this PR.