perf(hip): compile DFlash GPU draft top-K kernel for HIP

[cheese-cakee]

Summary

The DFlash GPU draft top-K + log-prob kernel (geometric_draft_topk_cuda.cu, added in #434) was compiled only on the CUDA backend. On HIP/ROCm builds DFLASH27B_HAVE_DRAFT_TOPK_CUDA was never defined, so Qwen35DFlashTarget::project_hidden_to_topk always took the CPU fallback: a full vocab x n_tokens device-to-host logits copy plus a CPU heap extract on every draft step. On a bandwidth-bound card that per-step D2H is a plausible contributor to the gfx1100 "~2x slower than llama.cpp" report in #457.

This compiles the exact same kernel for HIP. The body is pure arithmetic (no tensor cores, no CUDA-only intrinsics), so it builds unchanged through the existing hip_compat shim, the same way flashprefill_kernels.hip.cu is built.

Changes

• server/src/common/geometric_draft_topk.hip.cu (new): a thin TU that #includes the shared kernel body and is compiled with LANGUAGE HIP + -I hip_compat. On CUDA builds the .cu is still added directly; on HIP only this wrapper is.
• server/hip_compat/cuda_runtime.h: add the three cuda* -> hip* mappings the kernel needs and the shim did not yet cover (cudaPointerAttributes, cudaPointerGetAttributes, cudaMemoryTypeDevice).
• Rename the guard macro DFLASH27B_HAVE_DRAFT_TOPK_CUDA -> backend-neutral DFLASH27B_HAVE_DRAFT_TOPK, defined on both backends; update the two call sites (qwen35_dflash_target.cpp, test_dflash.cpp).
• server/CMakeLists.txt: register test_draft_topk_cuda on the HIP backend too (GPU-vs-CPU parity test), mirroring the existing HIP test_flashprefill_kernels wiring.
• README.md: correct the note that both flags are CUDA-only.

How it works

DFLASH_GPU_DRAFT_TOPK now takes effect on HIP: the kernel runs top-K + online-logsumexp directly on the logits device buffer and copies back only the n_positions x K results, eliminating the per-step full-vocab D2H and CPU heap extract. Compiling one shared body for both backends keeps the GPU result bit-for-bit identical to the CUDA build and to the CPU reference it is validated against; the automatic CPU fallback on any device failure (out-of-range index, allocation failure, etc.) is unchanged, so the change is output-neutral by construction.

Note on the sibling flag: DFLASH_GPU_VERIFY_ARGMAX already reads an in-graph ggml_argmax node in verify_tree and was never gated on the CUDA-only macro, so it needs no port and already runs on HIP. The README is corrected accordingly.

Performance

Not yet measured. The change removes a vocab x n_tokens (Qwen3.5 vocab ~152k) D2H plus a CPU heap top-K per draft step; the win scales with how bandwidth-bound the card is. A/B recipe on an AMD card: baseline DFLASH_GPU_DRAFT_TOPK=0, treatment DFLASH_GPU_DRAFT_TOPK=1, python server/scripts/bench_llm.py --bench HumanEval (temp 0), comparing decode tok/s and acceptance length.

Limitations

• The hipPointerAttribute_t.type member is spelled type on ROCm >= 5.0. All AMD targets in scope (ROCm 6.4.x on the gfx1151 CI runner, 7.x per the discrete-RDNA3 guidance in gfx1100 (RX 7900 XTX, ROCm 6.4.1): dflash_server ~2x slower than llama.cpp, DDTree speculation never engages, SIGSEGV at long context #457) use type; older ROCm that only exposes .memoryType is not supported.
• This is the qwen35/DFlash draft path only. It does not touch the laguna tree-verify HIP top_k (fix(spec-decode): DFlash tree-verify under KVFlash (qwen35 + laguna) + laguna ddtree HIP top_k #436) or the CUDA fused sampler (perf(cuda): port sample_logits chain to a fused GPU sampler kernel #478), which are different code paths.

Verification

• Static: no stale DFLASH27B_HAVE_DRAFT_TOPK_CUDA references remain; guard macro resolves on both backends; new HIP TU and test registration mirror the shipped flashprefill HIP pattern.
• Not build-validated locally this PR: no GPU toolchain on the authoring machine, and the CUDA validation box was unreachable while preparing this. The CUDA arm change is a macro rename plus a CMake comment, so a clean CUDA rebuild should be a no-op; the HIP arm (compile + the GPU-vs-CPU parity test test_draft_topk_cuda, and DFLASH_GPU_VERIFY_ARGMAX=2 dual-path mismatch mode) needs gfx1100/gfx1151. Requesting the gpu-tests-amd runner build (it compiles the tree) or a maintainer proxy to confirm the HIP compile and parity before merge.

[cubic-dev-ai]

1 issue found across 8 files

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cheese-cakee]

Rebased onto current main (453427d) to resolve the conflicts, and validated the HIP path on real AMD hardware.

On-HW validation (Radeon 8060S, gfx1151, wave32, ROCm 6.4.4): built test_draft_topk_cuda with LANGUAGE HIP (arch gfx1151, Release) and ran it on the GPU. The .hip.cu kernel now actually executes (previously the test skipped for lack of a device), and every case matches the CPU reference:

[PASS] n=15 vocab=151936 K=8 temp=1.00  id_mismatch=0 tie_swap=0 max_lp_err=3.166e-04
[PASS] n=1  vocab=151936 K=8 temp=1.00  id_mismatch=0 tie_swap=0 max_lp_err=1.488e-04
[PASS] n=15 vocab=151936 K=8 temp=0.70  id_mismatch=0 tie_swap=0 max_lp_err=9.727e-05
[PASS] n=15 vocab=151936 K=8 temp=2.00  id_mismatch=0 tie_swap=0 max_lp_err=3.815e-05
[PASS] n=7  vocab=1024   K=8 temp=1.00  id_mismatch=0 tie_swap=0 max_lp_err=9.537e-07
[PASS] n=32 vocab=4096   K=8 temp=1.00  id_mismatch=0 tie_swap=0 max_lp_err=1.335e-05
[PASS] n=3  vocab=257    K=8 temp=1.00  id_mismatch=0 tie_swap=0 max_lp_err=9.537e-07
[PASS] n=1  vocab=151936 K=1 temp=1.00  id_mismatch=0 tie_swap=0 max_lp_err=1.659e-04
[PASS] n=15 vocab=151936 K=4 temp=1.00  id_mismatch=0 tie_swap=0 max_lp_err=2.728e-04
[PASS] fallback contract: K=64 (>kMaxK) returned false
ALL PASS: 10/10 cases

All id_mismatch=0 and max_lp_err <= 3.17e-04 (tolerance 2e-3), including the Qwen3.5 vocab (151936) K=8 decode shape, and the CPU-fallback contract for K beyond the kernel range holds. The change stays output-neutral by construction (automatic CPU fallback on any failure).

[cubic-dev-ai]

No issues found across 8 files

_{Re-trigger cubic}

[howard0su]

suggest to cleanup. there are lots of CU files are shared between HIP and CUDA. We should not create hip.cu unless no other approach. This new pattern can confuse people.

[howard0su]

shall we have a separate PR to remove this knob as it was default already? DFLASH27B_HAVE_DRAFT_TOPK

[cheese-cakee]

in my opinion better as a separate PR so this one stays scoped to the HIP port