spec(rocm): BACKEND-ROCM GDN kernel family — spike + claim (gfx1100, issue #41) - #334
Conversation
…spike (mudler#41) The Qwen3.5-0.8B M2 blocker on discrete ROCm (throws at op 77; the reference tier cannot install on a dGPU) needs ten GDN ops with no ROCm kernel. This commit is the spike spec and the claim; implementation PRs follow one family per PR in throw-order (state-IO, conv, norm-gate, core, fused). Pre-claim validation on 4x gfx1100 / ROCm 7.14 (scratch, recorded in the spec): all ten kernels hand-translated from the CUDA donors and 68/68 standalone checks green (state paths and dtype conversions bit-exact vs an independent host reference); the drop-in TU compiles with the exact production flags; five red-first cross-device cases run 16/16 against the real library with unregistered ops skipping correctly. The key design finding: registering the full indexed set flips IndexedGdnOpsNative(kROCM) true, so the model takes the CUDA lane's device-resident path with zero model or runner edits. M2 disposition follows the mudler#269/mudler#273-ratified distributional gate for the near-tie regime. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: pi:kimi-k3 [pi]
…1 of the gfx1100 GDN slice (mudler#41) The current Qwen3.5-0.8B M2 throw on discrete ROCm (op 77). Hand-translated from the cuda_gdn.cu:248-335 donors, gated by the new cross-device case: bit-exact vs the CPU oracle across f32/f16/bf16 cache arms, uniform and spec-widened layouts, and the has_initial_state i8/i32/absent forms (the bar is byte equality — indexed data movement plus the boundary RNE conversion both sides share). Evidence (4x gfx1100, ROCm 7.14, Release): - test_backend_cross_device GDN state case: 48/48 assertions (runs, not skips) - ctest -R 'rocm|cross_device': 3/3 - full ctest: same 7 failures as the unmodified base 5812b8b (pre-existing host/lane state — async-scheduling expectations on the ROCm lane and the GDN model tests that fail AT op 77 on base; on this branch the test_loaded_engine_dense throw advances 77 -> op 6) - M2 rerun (Qwen3.5-0.8B): op 77 now resolves vt-native; the throw advances to op 5 (kCausalConv1dFwd, family 2) FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: pi:kimi-k3 [pi]
|
Family 1 (GDN-STATE-IO, ops 77/78) landed on this branch (
Next: family 2 (conv, ops 5/6) on a stacked branch. |
localai-org-maint-bot
left a comment
There was a problem hiding this comment.
@VikashLoomba The implementation/evidence looks carefully scoped, but this code checkpoint is missing the mandatory same-change record updates from AGENTS.md: docs/STATUS.md, docs/BENCHMARKS.md, the owning backend/kernel matrix row(s), roadmap roll-up where state changes, and an append-only evidence/state entry. docs/FEATURES.md/docs/USAGE.md do not replace those checkpoint surfaces. Please add the exact current lifecycle, honest no-speed/pending disposition, implementation/test anchors, and the gfx1100 evidence before this stack layer merges.
…x1100 (mudler#41) Updates the claim row + NOW.md to the landed state: all five GDN families pushed as stacked draft PRs (mudler#334/mudler#336/mudler#341/mudler#343/mudler#345), Qwen3.5-0.8B running end to end all-native on discrete gfx1100, and the two named follow-ups (MoeRouterTopK bf16-logits arm; GDN-path divergence characterization). FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: pi:kimi-k3 [pi]
|
The whole stack (#334, #336, #341, #343, #345, #348) landed on main, each as its own merge commit so your per-family commits and authorship stay intact. It reviewed well and the reason is the shape of it. Five new ROCm-only TUs compiled solely under The one shared-seam change is the right shape. Conflicts on landing were all additive-vs-additive and resolved by union: the ROCm source list in Two follow-ups, neither blocking:
Reviewed and landed with Claude Code. |
kCausalConv1dFwd / kCausalConv1dUpdate for ROCm (rocm_gdn_conv.hip), family 2 of the gfx1100 GDN slice, hand-translated from the CUDA donors. Conflicts resolved by union, all additive-vs-additive: the ROCm source list in CMakeLists.txt gained rocm_sample.hip (#234, already on main) and rocm_gdn_state.hip (#334) independently, and both the FEATURES ROCm row and the coordination claim table are keyed records where the two sides touch different keys. Refs #336, #41. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Claude:claude-opus-5 [ClaudeCode]
The MoE-path ops the GDN slice surfaced as the named blocker once the hybrid model ran: MoeRouterTopK gains a bf16-logits arm and MoeSiluMul lands, so the MoE half of Qwen3.5-0.8B stays on device instead of falling back. Stacked on the five GDN PRs (#334/#336/#341/#343/#345), all landed here. ROCm-only kernels plus additive registrations and cross-device cases. Conflicts resolved by union (additive vs additive). Refs #348, #41. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Claude:claude-opus-5 [ClaudeCode]
The GDN slice's NOW.md row pushed the file to 6019 characters, over the 6000-character budget check-now-current enforces (in agent-preflight and in CI). The row keeps what changed — #140, gfx1200 M0-M4 (#269), the GDN slice and MoE ops (#334-#348), Qwen3.5-0.8B running e2e all-native — and drops the restatement; the next-step cell names the two things actually open, strict CPU parity and the GDN drift characterization. python3 scripts/check-now-current.py OK (6019 > 6000 before) Refs #41. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: Claude:claude-opus-5 [ClaudeCode]
…Rs conflict (#364) (#368) * spec(record): shared record surfaces are a lock, and the lock is the conflicts (#364) 16 of 29 open PRs are CONFLICTING, and 13 of those 16 conflict in bookkeeping files ONLY, with no product code involved. Measured at origin/main d928e2c by running `git merge-tree --write-tree` against every open PR head: coordination.md in 8, NOW.md in 5, roadmap_v1.md in 4, check-public-doc-tables.py in 4, STATUS.md in 4 -- and any src/ or tests/ path in just 3, one of which is a stale mega-branch. Three surfaces are shaped so concurrent PRs must collide, rather than merely risk it. NOW.md is a fixed-size shared buffer sitting at EXACTLY 6000/6000 chars, so adding a row requires evicting another and every PR becomes a read-modify-write of one global -- where the conflict is the lucky outcome, since a clean three-way merge would apply both evictions and both additions and silently drop live rows. STATUS_RATCHET is a hardcoded byte count of a different file that may only fall, so a PR owing STATUS.md one lifecycle line must delete unrelated prose to pay for it; the checker's own comment already records the failure and answered it with slack rather than by removing the coupling. And coordination.md's claims table is insert-at-one-anchor, which is why six ROCm GDN PRs from one author's sequential stack conflict on nothing else. It also contradicts the protocol it serves. AGENTS.md holds that history is git and that there is no state log, yet both claims tables are state logs duplicating `gh pr list`, row/<ID> branch names and issue state. The argument that refuses a waiver registry applies unchanged to a claims registry, and this is the third instance of one failure mode: policy.csv was retired in 0f3e44e and the per-class line budgets on 2026-08-10, both because the gate fired on ordinary work. The exonerated surfaces share one property, one writer per file. Specs are one file per row and took ZERO conflicts across the whole sample, so W1-W5 rewrite the three offenders into that shape or derive them, keep every local per-cell and per-paragraph cap, and record the invariant: no surface that every PR must write. This commit is the spec and the record only. No checker semantic, no doc content and no product source changes; the four counter re-pins below are the existing bump convention for a real new row, each carrying its reason. Filing this one row tripped four separate hardcoded global counters (ENGINE_ROWS, two engine-matrix summary roll-ups, and RUNNABLE_BASELINE), which is the same defect the row exists to remove and is recorded here as evidence rather than worked around. Issue: #364 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code] * fix(record): remove the two global counters that made every PR a shared write (#364) W1, W3, W4 and W5 of the spec. The claims table (W2) follows separately. W1 removes the `chars` key of STATUS_RATCHET. It was a byte count of docs/STATUS.md stored in scripts/check-public-doc-tables.py and allowed to move only downward, so a PR owing the page one lifecycle line had to delete unrelated prose from another row to pay for it AND re-pin the constant -- two shared files per PR, both of which every other PR was also editing. RED-BEFORE measured: the live page was EXACTLY 243245 of 243245 permitted chars, and one ordinary capability line put it 27 over. The cut is narrower than the spec's W1 text, deliberately. The four keys are not alike: `chars` is a LENGTH and moves on every edit, while `h2_sections`, `long_paragraphs` and `oversized_cells` count QUALITY DEFECTS, so an ordinary line moves none of them and two concurrent PRs do not collide on them. Those three are kept and carry the whole anti-decay obligation -- BENCHMARKS.md's 11,127-line decay would have tripped two of them on its own. Deleting them would have dropped a real obligation for no conflict benefit, which is the case the spec's stop condition names. W3 removes MAX_CHARS from check-now-current.py. The tracked digest measured EXACTLY 6000 of 6000: tuned to the byte, no headroom, so a row could only be added by evicting one. MAX_LINES and MAX_ENTRY_CHARS stay and carry the obligation -- a row costs one line rather than a variable number of bytes, and each entry is still capped locally. W4 sorts the roadmap issue table by stable row ID. The 43 rows are reordered and otherwise byte-identical, verified as a set before and after. Appending at one anchor is why concurrent additions collided; sorting puts them in different hunks. W5 records the invariant in AGENTS.md: no surface that every PR must write, and the three shapes that are admissible instead. The new tests/scripts/test_record_merge_shape.py asks git the question directly rather than inspecting a checker, and it CORRECTED a claim I had made in the spec. Removing a budget does NOT on its own make two row additions merge -- two appends collide at the same anchor either way. What the budget added on top was a forced deletion of unrelated content, and that is the part with teeth: it made every PR edit lines it did not own, and it made a SUCCESSFUL merge unsafe, since git resolving two such branches applies both evictions and both victims vanish with no gate noticing. W3 removes the eviction, W4 removes the collision, and neither alone is sufficient. The spec is corrected in the same commit. Gates: preflight rc=0; check_public_doc_tables 52 OK, check_now_current 11 OK, record_merge_shape 8 OK; check-public-doc-tables, check-now-current and check-agent-record all green on the live tree. No product source is touched. Issue: #364 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code] * fix(record): a claim gets its own file, so concurrent claims stop colliding (#364) W2, and the largest single conflict source in the repository: the claims TABLE in coordination.md is insert-at-one-anchor, so every concurrent claim appends a row at the same line. It conflicted in 8 of the 16 conflicting open PRs measured at origin/main d928e2c, six of them (#334 #336 #341 #343 #345 #348) one author's sequential ROCm GDN stack whose ONLY conflict was this file. Their work did not overlap at all. New claims now go in .agents/claims/CLAIM-<ID>.md, one file per claim. parse_active_claims reads that directory AND the legacy table, so both are valid to the gate and the bidirectional cross-check is unchanged. A file with one writer cannot collide -- the shape .agents/specs/ already has, which is why specs took ZERO conflicts across the measured sample. The design differs from the spec's W2, which said to derive claims from open PRs and branch names. Reading the consumers showed that to be wrong twice over, and the spec is corrected rather than the change quietly reinterpreted. Deriving would put a NETWORK CALL inside an offline gate -- check-agent-record.py runs in preflight with no network -- and the table is not redundant with the PR list: check_row_contracts uses it for a bidirectional cross-check, where every SPIKE/ACTIVE matrix row must be claimed by a live claim and every claim must name rows in those states. A PR list cannot supply that, so deriving would have dropped a real guarantee. That is the case the spec's own stop condition names. Deliberately ADDITIVE: not one existing row is migrated. The 115 rows are spread across several interleaved tables, struck-through released entries and prose blocks, some below a later heading -- bulk-rewriting that is how a record gets lost, in a diff no reviewer could check. Rows leave as their claims close and the table empties on its own, while the conflict source is closed for every future claim from the day this lands. RED-BEFORE proven by mutation: with claim_sources reverted to coordination.md alone, tests/scripts/test_record_merge_shape.py goes 2 failures; restored byte-for-byte (md5 b9f325f1238a954001132a19309e7caa before and after) and green again at 11/11. Gates: preflight green apart from the commit-trailer repair in the same push; check-agent-record, claim-view --check-local, check-pr-size 34 all green. Issue: #364 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code] * test(record): mutation evidence in each changed checker's own suite (#364) check-pr-size.py was RIGHT and this is its finding, not a workaround. Three governance checkers changed in this row -- check-agent-record.py (per-claim file source), check-pr-size.py (the .agents/claims path class) and check-gate-commands.py (the RUNNABLE_BASELINE re-pin) -- and the evidence for all three lived in the NEW tests/scripts/test_record_merge_shape.py rather than in each checker's paired suite. The gate requires the paired suite, correctly: that is where a later reader of that checker looks, and a new file is easy to miss or delete wholesale. Added, each with a mutation that proves the assertion is load-bearing: test_agent_record.py -- a claim in its own file registers exactly as a table row does; patching claim_sources back to coordination.md alone HIDES it (the mutation); a claim declared twice across the two sources is still rejected, so reading a second source did not weaken the duplicate check. test_check_pr_size.py -- .agents/claims/CLAIM-*.md classifies as the same class as the per-row spec it mirrors; substituting a never-matching pattern for CLAIM makes classify_path raise (the mutation), proving the clause is not shadowed by a broader rule; and a non-markdown path in the directory still fails closed, so the pattern is not a blanket directory exemption -- which AGENTS.md forbids. test_check_gate_commands.py -- the row is credited runnable and its spec really names failable commands; removing the entry from RUNNABLE_BASELINE must break the exact pin (the mutation), so the baseline is not decorative. Also moved `if __name__ == "__main__"` to the END of test_agent_record.py and test_check_pr_size.py. It sat mid-file, so classes appended after it were defined AFTER unittest.main() had already collected and run -- the suites reported OK while never executing the new tests (34 and 25 before, 38 and 29 after). A green suite that silently skips what was just added is the same class of defect as the rest of this row. Suites: test_agent_record 29, test_check_pr_size 38, test_check_gate_commands 27, test_record_merge_shape 11, all OK. Issue: #364 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code] --------- Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Row
BACKEND-ROCM— M3 GDN kernel slice (issue #41). Claim:CLAIM-ROCM-GDN-KERNELS.Before starting
BACKEND-ROCM(backend-matrix), roadmap issue-table row for ROCm (AMD GPU) backend #41.src/vt/rocm/rocm_ops.hip(no GDN registrations on5812b8b6),src/vt/cuda/cuda_gdn.cu+cuda_ops.cudonors,src/vt/cpu/cpu_ops.cpporacles,qwen3_5.cppIndexedGdnOpsNative,include/vt/ops.hFn aliases,src/vt/ops.cppvalidators.What changed
This PR opens with the spike spec + claim (
.agents/specs/rocm-gdn-kernels.md, the coordination claim row, the NOW.md line) for the ten GDN ops Qwen3.5-0.8B needs on discrete ROCm:kGdnStateGather/Scatter(77/78),kCausalConv1dFwd/Update(5/6),kRmsNormGated(9),kSigmoidGateBf16(58),kGdnPrefill/Decode(10/11),kGdnPostConv(65),kAttnQkNormRopeGate(67). Implementation lands on this branch one family per commit in throw-order (state-IO first — the current M2 throw), each with its red-first cross-device case, focused + full gate, and an M2 rerun. The design finding that shapes it: registering the full indexed set flipsIndexedGdnOpsNative(kROCM)true, so the model takes the CUDA lane's device-resident path with zero model/runner edits. Perf machinery (WMMA chunked prefill, fused/packed decode, register/tiled conv, Triton AOT, spec/fp8 variants) deliberately unported per docs/ROCM.md §6 — the portable scan serves prefill and decode.Evidence
scripts/agent-preflight.shpasses — green in the worktree (exit 0). Two host-environment suites need per-process mitigations on this box, both pre-existing on an unmodified base and unrelated to the change:test_agent_onboard(host gitinit.defaultBranch=mainvs the test's hardcodedmaster; run withGIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=init.defaultBranch GIT_CONFIG_VALUE_0=master) andtrailer suites(site-packagestestsshadows the repo namespace; temporarytests/__init__.py+tests/scripts/__init__.pyduring the run). Both verified failing identically on the clean base before my edit.ACTIVE);NOW.md+coordination.mdcarry the claim.Pre-claim validation (on 4× gfx1100 / ROCm 7.14, recorded in the spec): all ten kernels hand-translated from the CUDA donors and validated in standalone harnesses against independent host references — 68/68 checks (state paths and dtype conversions bit-exact; compute at expf/FMA ulp level). The drop-in TU compiles with the exact production flags (
-O3 -ffp-contract=off -std=c++20 --offload-arch=gfx1100). The five cross-device cases were run assembled into a scratch copy of the test file against the reallibvllm.a: 16/16 green, GDN cases skipping unregistered ops as designed.Speed claims
Honest gaps