Skip to content

spec(rocm): BACKEND-ROCM GDN kernel family — spike + claim (gfx1100, issue #41) - #334

Merged
localai-bot merged 3 commits into
mudler:mainfrom
VikashLoomba:row/ROCM-GDN-KERNELS
Aug 11, 2026
Merged

spec(rocm): BACKEND-ROCM GDN kernel family — spike + claim (gfx1100, issue #41)#334
localai-bot merged 3 commits into
mudler:mainfrom
VikashLoomba:row/ROCM-GDN-KERNELS

Conversation

@VikashLoomba

Copy link
Copy Markdown
Contributor

Row

BACKEND-ROCM — M3 GDN kernel slice (issue #41). Claim: CLAIM-ROCM-GDN-KERNELS.

Before starting

What changed

This PR opens with the spike spec + claim (.agents/specs/rocm-gdn-kernels.md, the coordination claim row, the NOW.md line) for the ten GDN ops Qwen3.5-0.8B needs on discrete ROCm: kGdnStateGather/Scatter (77/78), kCausalConv1dFwd/Update (5/6), kRmsNormGated (9), kSigmoidGateBf16 (58), kGdnPrefill/Decode (10/11), kGdnPostConv (65), kAttnQkNormRopeGate (67). Implementation lands on this branch one family per commit in throw-order (state-IO first — the current M2 throw), each with its red-first cross-device case, focused + full gate, and an M2 rerun. The design finding that shapes it: registering the full indexed set flips IndexedGdnOpsNative(kROCM) true, so the model takes the CUDA lane's device-resident path with zero model/runner edits. Perf machinery (WMMA chunked prefill, fused/packed decode, register/tiled conv, Triton AOT, spec/fp8 variants) deliberately unported per docs/ROCM.md §6 — the portable scan serves prefill and decode.

Evidence

  • scripts/agent-preflight.sh passes — green in the worktree (exit 0). Two host-environment suites need per-process mitigations on this box, both pre-existing on an unmodified base and unrelated to the change: test_agent_onboard (host git init.defaultBranch=main vs the test's hardcoded master; run with GIT_CONFIG_COUNT=1 GIT_CONFIG_KEY_0=init.defaultBranch GIT_CONFIG_VALUE_0=master) and trailer suites (site-packages tests shadows the repo namespace; temporary tests/__init__.py + tests/scripts/__init__.py during the run). Both verified failing identically on the clean base before my edit.
  • tests that cover this change: this commit is records-only (spec/claim/NOW). The GDN cross-device cases land with their family commits, red-first.
  • same-change doc obligations: no lifecycle change (the row stays ACTIVE); NOW.md + coordination.md carry the claim.

Pre-claim validation (on 4× gfx1100 / ROCm 7.14, recorded in the spec): all ten kernels hand-translated from the CUDA donors and validated in standalone harnesses against independent host references — 68/68 checks (state paths and dtype conversions bit-exact; compute at expf/FMA ulp level). The drop-in TU compiles with the exact production flags (-O3 -ffp-contract=off -std=c++20 --offload-arch=gfx1100). The five cross-device cases were run assembled into a scratch copy of the test file against the real libvllm.a: 16/16 green, GDN cases skipping unregistered ops as designed.

Speed claims

  • This PR makes NO speed claim.

Honest gaps

…spike (mudler#41)

The Qwen3.5-0.8B M2 blocker on discrete ROCm (throws at op 77; the reference
tier cannot install on a dGPU) needs ten GDN ops with no ROCm kernel. This
commit is the spike spec and the claim; implementation PRs follow one family
per PR in throw-order (state-IO, conv, norm-gate, core, fused).

Pre-claim validation on 4x gfx1100 / ROCm 7.14 (scratch, recorded in the
spec): all ten kernels hand-translated from the CUDA donors and 68/68
standalone checks green (state paths and dtype conversions bit-exact vs an
independent host reference); the drop-in TU compiles with the exact
production flags; five red-first cross-device cases run 16/16 against the
real library with unregistered ops skipping correctly. The key design
finding: registering the full indexed set flips IndexedGdnOpsNative(kROCM)
true, so the model takes the CUDA lane's device-resident path with zero
model or runner edits. M2 disposition follows the mudler#269/mudler#273-ratified
distributional gate for the near-tie regime.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: pi:kimi-k3 [pi]
…1 of the gfx1100 GDN slice (mudler#41)

The current Qwen3.5-0.8B M2 throw on discrete ROCm (op 77). Hand-translated
from the cuda_gdn.cu:248-335 donors, gated by the new cross-device case:
bit-exact vs the CPU oracle across f32/f16/bf16 cache arms, uniform and
spec-widened layouts, and the has_initial_state i8/i32/absent forms (the bar
is byte equality — indexed data movement plus the boundary RNE conversion
both sides share).

Evidence (4x gfx1100, ROCm 7.14, Release):
- test_backend_cross_device GDN state case: 48/48 assertions (runs, not skips)
- ctest -R 'rocm|cross_device': 3/3
- full ctest: same 7 failures as the unmodified base 5812b8b (pre-existing
  host/lane state — async-scheduling expectations on the ROCm lane and the
  GDN model tests that fail AT op 77 on base; on this branch the
  test_loaded_engine_dense throw advances 77 -> op 6)
- M2 rerun (Qwen3.5-0.8B): op 77 now resolves vt-native; the throw advances
  to op 5 (kCausalConv1dFwd, family 2)

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: pi:kimi-k3 [pi]
@VikashLoomba

Copy link
Copy Markdown
Contributor Author

Family 1 (GDN-STATE-IO, ops 77/78) landed on this branch (152c9e0f), gates run on 4× gfx1100 / ROCm 7.14, Release:

Gate Result
New cross-device case (GDN state gather/scatter) 48/48 assertions, runs not skips — bit-exact vs CPU oracle over f32/f16/bf16 cache arms × uniform/widened × has-i8/i32/absent
ctest -R 'rocm|cross_device' 3/3
Full ctest same 7 failures as unmodified base 5812b8b6 (pre-existing lane state — async-scheduling expectations + the GDN model tests that fail AT op 77 on base); delta zero
In-vivo advance test_loaded_engine_dense throw moves 77 → op 6; Qwen3.5-0.8B M2 rerun: op 77 resolves vt-native, throw moves 77 → op 5

agent-preflight.sh --staged green (with the two host-env mitigations noted in the body); check-commit-trailers green; the doc-checkpoint gate drove truthful docs/USAGE.md + docs/FEATURES.md updates (the FEATURES row was stale post-#140 — now reflects the op set + the #269 near-tie disposition).

Next: family 2 (conv, ops 5/6) on a stacked branch.

@localai-org-maint-bot localai-org-maint-bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@VikashLoomba The implementation/evidence looks carefully scoped, but this code checkpoint is missing the mandatory same-change record updates from AGENTS.md: docs/STATUS.md, docs/BENCHMARKS.md, the owning backend/kernel matrix row(s), roadmap roll-up where state changes, and an append-only evidence/state entry. docs/FEATURES.md/docs/USAGE.md do not replace those checkpoint surfaces. Please add the exact current lifecycle, honest no-speed/pending disposition, implementation/test anchors, and the gfx1100 evidence before this stack layer merges.

…x1100 (mudler#41)

Updates the claim row + NOW.md to the landed state: all five GDN families
pushed as stacked draft PRs (mudler#334/mudler#336/mudler#341/mudler#343/mudler#345), Qwen3.5-0.8B running
end to end all-native on discrete gfx1100, and the two named follow-ups
(MoeRouterTopK bf16-logits arm; GDN-path divergence characterization).

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: pi:kimi-k3 [pi]
@localai-bot

Copy link
Copy Markdown
Collaborator

The whole stack (#334, #336, #341, #343, #345, #348) landed on main, each as its own merge commit so your per-family commits and authorship stay intact.

It reviewed well and the reason is the shape of it. Five new ROCm-only TUs compiled solely under VLLM_CPP_HIP, additive registrations in rocm_ops.hip, additive cases in test_backend_cross_device.cpp — so a backend that does not register the ops just skips them, and CUDA/CPU/Vulkan paths are untouched by construction rather than by care. Splitting it into one family per PR made each one reviewable on its own, which is not free work and it showed.

The one shared-seam change is the right shape. CheckGdnCommon's compressed-state clause read q.device.type == DeviceType::kCUDA; asking Backend::SupportsCompressedGdnState() instead means CUDA answers true and takes exactly the branch it took before, while the op layer stops naming a device. That it mirrors the SupportsCompressedConvState seam already sitting next to it is what makes it a fit rather than a new pattern.

Conflicts on landing were all additive-vs-additive and resolved by union: the ROCm source list in CMakeLists.txt had independently gained rocm_sample.hip (#234) and your rocm_gdn_state.hip; the coordination.md claim table had two different claim rows; the FEATURES ROCm row was rewritten by both this stack and #273. Verified after: no duplicate OpId:: registrations, no duplicate declarations, no markers.

Two follow-ups, neither blocking:

  1. The FEATURES ROCm row in the Backend × framework table got a row shaped for the gap table below it — two prose cells instead of one status cell plus three framework ticks. The columns shifted by one, so vLLM's ✅ became prose, SGLang inherited vLLM's, and llama.cpp's fell off the end; the table briefly claimed less ROCm support for two other projects. Repaired on landing, and both ROCm rows also had to come inside check-public-doc-tables' 220-char cell budget and its no-em-dash rule. Easy to trip — the two ROCm rows look alike and are not the same table.

  2. Your own record says it: strict CPU parity is not met and the GDN-path divergence is characterized as not-state-rounding but otherwise open. That is carried forward verbatim into FEATURES and NOW rather than rounded up, so the next person reads the same claim you made.

Reviewed and landed with Claude Code.

@localai-bot
localai-bot merged commit 29a32c8 into mudler:main Aug 11, 2026
localai-bot pushed a commit that referenced this pull request Aug 11, 2026
kCausalConv1dFwd / kCausalConv1dUpdate for ROCm (rocm_gdn_conv.hip), family 2
of the gfx1100 GDN slice, hand-translated from the CUDA donors.

Conflicts resolved by union, all additive-vs-additive: the ROCm source list in
CMakeLists.txt gained rocm_sample.hip (#234, already on main) and
rocm_gdn_state.hip (#334) independently, and both the FEATURES ROCm row and the
coordination claim table are keyed records where the two sides touch different
keys.

Refs #336, #41.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Claude:claude-opus-5 [ClaudeCode]
localai-bot pushed a commit that referenced this pull request Aug 11, 2026
The MoE-path ops the GDN slice surfaced as the named blocker once the hybrid
model ran: MoeRouterTopK gains a bf16-logits arm and MoeSiluMul lands, so the
MoE half of Qwen3.5-0.8B stays on device instead of falling back.

Stacked on the five GDN PRs (#334/#336/#341/#343/#345), all landed here.
ROCm-only kernels plus additive registrations and cross-device cases.
Conflicts resolved by union (additive vs additive).

Refs #348, #41.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Claude:claude-opus-5 [ClaudeCode]
localai-bot pushed a commit that referenced this pull request Aug 11, 2026
The GDN slice's NOW.md row pushed the file to 6019 characters, over the
6000-character budget check-now-current enforces (in agent-preflight and in CI).
The row keeps what changed — #140, gfx1200 M0-M4 (#269), the GDN slice and MoE
ops (#334-#348), Qwen3.5-0.8B running e2e all-native — and drops the
restatement; the next-step cell names the two things actually open, strict CPU
parity and the GDN drift characterization.

  python3 scripts/check-now-current.py   OK  (6019 > 6000 before)

Refs #41.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: Claude:claude-opus-5 [ClaudeCode]
mudler added a commit that referenced this pull request Aug 11, 2026
…Rs conflict (#364) (#368)

* spec(record): shared record surfaces are a lock, and the lock is the conflicts (#364)

16 of 29 open PRs are CONFLICTING, and 13 of those 16 conflict in bookkeeping
files ONLY, with no product code involved. Measured at origin/main d928e2c by
running `git merge-tree --write-tree` against every open PR head:
coordination.md in 8, NOW.md in 5, roadmap_v1.md in 4,
check-public-doc-tables.py in 4, STATUS.md in 4 -- and any src/ or tests/ path
in just 3, one of which is a stale mega-branch.

Three surfaces are shaped so concurrent PRs must collide, rather than merely
risk it. NOW.md is a fixed-size shared buffer sitting at EXACTLY 6000/6000
chars, so adding a row requires evicting another and every PR becomes a
read-modify-write of one global -- where the conflict is the lucky outcome,
since a clean three-way merge would apply both evictions and both additions and
silently drop live rows. STATUS_RATCHET is a hardcoded byte count of a
different file that may only fall, so a PR owing STATUS.md one lifecycle line
must delete unrelated prose to pay for it; the checker's own comment already
records the failure and answered it with slack rather than by removing the
coupling. And coordination.md's claims table is insert-at-one-anchor, which is
why six ROCm GDN PRs from one author's sequential stack conflict on nothing
else.

It also contradicts the protocol it serves. AGENTS.md holds that history is git
and that there is no state log, yet both claims tables are state logs
duplicating `gh pr list`, row/<ID> branch names and issue state. The argument
that refuses a waiver registry applies unchanged to a claims registry, and this
is the third instance of one failure mode: policy.csv was retired in 0f3e44e
and the per-class line budgets on 2026-08-10, both because the gate fired on
ordinary work.

The exonerated surfaces share one property, one writer per file. Specs are one
file per row and took ZERO conflicts across the whole sample, so W1-W5 rewrite
the three offenders into that shape or derive them, keep every local per-cell
and per-paragraph cap, and record the invariant: no surface that every PR must
write.

This commit is the spec and the record only. No checker semantic, no doc
content and no product source changes; the four counter re-pins below are the
existing bump convention for a real new row, each carrying its reason.

Filing this one row tripped four separate hardcoded global counters
(ENGINE_ROWS, two engine-matrix summary roll-ups, and RUNNABLE_BASELINE), which
is the same defect the row exists to remove and is recorded here as evidence
rather than worked around.

Issue: #364

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

* fix(record): remove the two global counters that made every PR a shared write (#364)

W1, W3, W4 and W5 of the spec. The claims table (W2) follows separately.

W1 removes the `chars` key of STATUS_RATCHET. It was a byte count of
docs/STATUS.md stored in scripts/check-public-doc-tables.py and allowed to move
only downward, so a PR owing the page one lifecycle line had to delete unrelated
prose from another row to pay for it AND re-pin the constant -- two shared files
per PR, both of which every other PR was also editing. RED-BEFORE measured: the
live page was EXACTLY 243245 of 243245 permitted chars, and one ordinary
capability line put it 27 over.

The cut is narrower than the spec's W1 text, deliberately. The four keys are not
alike: `chars` is a LENGTH and moves on every edit, while `h2_sections`,
`long_paragraphs` and `oversized_cells` count QUALITY DEFECTS, so an ordinary
line moves none of them and two concurrent PRs do not collide on them. Those
three are kept and carry the whole anti-decay obligation -- BENCHMARKS.md's
11,127-line decay would have tripped two of them on its own. Deleting them would
have dropped a real obligation for no conflict benefit, which is the case the
spec's stop condition names.

W3 removes MAX_CHARS from check-now-current.py. The tracked digest measured
EXACTLY 6000 of 6000: tuned to the byte, no headroom, so a row could only be
added by evicting one. MAX_LINES and MAX_ENTRY_CHARS stay and carry the
obligation -- a row costs one line rather than a variable number of bytes, and
each entry is still capped locally.

W4 sorts the roadmap issue table by stable row ID. The 43 rows are reordered and
otherwise byte-identical, verified as a set before and after. Appending at one
anchor is why concurrent additions collided; sorting puts them in different
hunks.

W5 records the invariant in AGENTS.md: no surface that every PR must write, and
the three shapes that are admissible instead.

The new tests/scripts/test_record_merge_shape.py asks git the question directly
rather than inspecting a checker, and it CORRECTED a claim I had made in the
spec. Removing a budget does NOT on its own make two row additions merge -- two
appends collide at the same anchor either way. What the budget added on top was
a forced deletion of unrelated content, and that is the part with teeth: it made
every PR edit lines it did not own, and it made a SUCCESSFUL merge unsafe, since
git resolving two such branches applies both evictions and both victims vanish
with no gate noticing. W3 removes the eviction, W4 removes the collision, and
neither alone is sufficient. The spec is corrected in the same commit.

Gates: preflight rc=0; check_public_doc_tables 52 OK, check_now_current 11 OK,
record_merge_shape 8 OK; check-public-doc-tables, check-now-current and
check-agent-record all green on the live tree. No product source is touched.

Issue: #364

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

* fix(record): a claim gets its own file, so concurrent claims stop colliding (#364)

W2, and the largest single conflict source in the repository: the claims TABLE in
coordination.md is insert-at-one-anchor, so every concurrent claim appends a row
at the same line. It conflicted in 8 of the 16 conflicting open PRs measured at
origin/main d928e2c, six of them (#334 #336 #341 #343 #345 #348) one author's
sequential ROCm GDN stack whose ONLY conflict was this file. Their work did not
overlap at all.

New claims now go in .agents/claims/CLAIM-<ID>.md, one file per claim.
parse_active_claims reads that directory AND the legacy table, so both are valid
to the gate and the bidirectional cross-check is unchanged. A file with one
writer cannot collide -- the shape .agents/specs/ already has, which is why specs
took ZERO conflicts across the measured sample.

The design differs from the spec's W2, which said to derive claims from open PRs
and branch names. Reading the consumers showed that to be wrong twice over, and
the spec is corrected rather than the change quietly reinterpreted. Deriving
would put a NETWORK CALL inside an offline gate -- check-agent-record.py runs in
preflight with no network -- and the table is not redundant with the PR list:
check_row_contracts uses it for a bidirectional cross-check, where every
SPIKE/ACTIVE matrix row must be claimed by a live claim and every claim must name
rows in those states. A PR list cannot supply that, so deriving would have
dropped a real guarantee. That is the case the spec's own stop condition names.

Deliberately ADDITIVE: not one existing row is migrated. The 115 rows are spread
across several interleaved tables, struck-through released entries and prose
blocks, some below a later heading -- bulk-rewriting that is how a record gets
lost, in a diff no reviewer could check. Rows leave as their claims close and the
table empties on its own, while the conflict source is closed for every future
claim from the day this lands.

RED-BEFORE proven by mutation: with claim_sources reverted to coordination.md
alone, tests/scripts/test_record_merge_shape.py goes 2 failures; restored
byte-for-byte (md5 b9f325f1238a954001132a19309e7caa before and after) and green
again at 11/11.

Gates: preflight green apart from the commit-trailer repair in the same push;
check-agent-record, claim-view --check-local, check-pr-size 34 all green.

Issue: #364

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

* test(record): mutation evidence in each changed checker's own suite (#364)

check-pr-size.py was RIGHT and this is its finding, not a workaround. Three
governance checkers changed in this row -- check-agent-record.py (per-claim file
source), check-pr-size.py (the .agents/claims path class) and
check-gate-commands.py (the RUNNABLE_BASELINE re-pin) -- and the evidence for
all three lived in the NEW tests/scripts/test_record_merge_shape.py rather than
in each checker's paired suite. The gate requires the paired suite, correctly:
that is where a later reader of that checker looks, and a new file is easy to
miss or delete wholesale.

Added, each with a mutation that proves the assertion is load-bearing:

test_agent_record.py -- a claim in its own file registers exactly as a table row
does; patching claim_sources back to coordination.md alone HIDES it (the
mutation); a claim declared twice across the two sources is still rejected, so
reading a second source did not weaken the duplicate check.

test_check_pr_size.py -- .agents/claims/CLAIM-*.md classifies as the same class
as the per-row spec it mirrors; substituting a never-matching pattern for CLAIM
makes classify_path raise (the mutation), proving the clause is not shadowed by
a broader rule; and a non-markdown path in the directory still fails closed, so
the pattern is not a blanket directory exemption -- which AGENTS.md forbids.

test_check_gate_commands.py -- the row is credited runnable and its spec really
names failable commands; removing the entry from RUNNABLE_BASELINE must break
the exact pin (the mutation), so the baseline is not decorative.

Also moved `if __name__ == "__main__"` to the END of test_agent_record.py and
test_check_pr_size.py. It sat mid-file, so classes appended after it were
defined AFTER unittest.main() had already collected and run -- the suites
reported OK while never executing the new tests (34 and 25 before, 38 and 29
after). A green suite that silently skips what was just added is the same class
of defect as the rest of this row.

Suites: test_agent_record 29, test_check_pr_size 38, test_check_gate_commands 27,
test_record_merge_shape 11, all OK.

Issue: #364

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:claude-opus-5 [Claude Code]

---------

Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants