Support bare HIP convergent warp and lane intrinsics

[Maou3434]

ISSUE HIP: bare warp intrinsics without _sync suffix

---

How the Issue is Fixed

Issue: HIP allows convergent warp/lane intrinsics (__shfl, __shfl_up, __shfl_down, __shfl_xor, __ballot, __any, __all) to be called without a mask argument (since the mask is implicit in convergent AMD wavefronts). Previously, only the _sync variants (which require a mask as their first argument) were recognized. Using the bare forms caused unknown-identifier errors.

Fix:

1. Frontend Recognition: The bare names are added to the built-ins table as aliases for their _sync counterparts.
2. Implicit Mask Insertion during Lowering: When lowering bare convergent forms, the middle-end IR lowerer synthetically inserts a default mask (0xFFFFFFFFu, representing all active lanes). This allows the rest of the compilation pipeline to handle them uniformly as masked operations.
3. Backend Robustness: The backends (NVIDIA, AMDGPU, CPU) are updated to correctly access the actual value or predicate operand by checking num_operands to determine if a mask is present (operand index 1 if a mask is present, or index 0 if not). Note: Previously, the backends had a bug where they incorrectly assumed that the value or predicate was always at operand 0 (which is the mask).

---

File-by-File Breakdown of Changes

1. src/fe/sema.c

• Why it was touched: To register the bare forms in the compiler.
• Changes:
  • Added the bare forms (__ballot, __any, __all, __shfl, __shfl_up, __shfl_down, __shfl_xor) to the static cuda_builtins list.
  • Updated check_expr to recognize bare versions (__ballot, __any, __all) when resolving the return types (rt), matching them alongside their _sync equivalents.

2. src/ir/bir_lower.c

• Why it was touched: To translate the bare frontend calls into BIR instructions with the implicit mask prepended.
• Changes:
  • Expanded stab (warp shuffle lookup table) and vtab (warp vote lookup table) to include bare functions with a sync flag (0 for bare, 1 for _sync).
  • If a bare variant is being lowered (!sync), a constant 0xFFFFFFFFu mask is synthetically generated and prepended as the first operand. The remaining function arguments (value, lane, etc.) are then appended normally.

3. src/nvidia/isel.c

• Why it was touched: To make sure it reads the correct operand indices, avoiding mapping the mask to the target registers.
• Changes:
  • In is_shfl, it now queries I->num_operands >= 3 to determine if a mask is present. If so, it reads the value and lane from indices 1 and 2 respectively, instead of assuming they are always at indices 0 and 1.
  • In is_vote, it queries I->num_operands > 1 to find the predicate at index 1 instead of index 0.

4. src/amdgpu/isel.c

• Why it was touched: To correctly extract the predicate operand in the presence of a mask.
• Changes:
  • Updated the cases BIR_BALLOT, BIR_VOTE_ANY, and BIR_VOTE_ALL to fetch the predicate from I->operands[I->num_operands > 1 ? 1 : 0], ensuring the correct predicate is resolved instead of the mask.

5. src/cpu/cpu_emit.c

• Why it was touched: To properly resolve operands for CPU-simulated shuffle and vote operations.
• Changes:
  • Updated BIR_SHFL / BIR_SHFL_UP / BIR_SHFL_DOWN / BIR_SHFL_XOR and BIR_VOTE_ANY / BIR_VOTE_ALL to load from operand 1 if num_operands > 1 (since the mask is prepended at index 0).

6. tests/tcomp.c & tests/test_shfl.cu

• Why they were touched: To add compiler test coverage validating that the bare forms successfully compile and select instructions without regressions.
• Changes:
  • Registered cmp_shfl in the compiler test runner list (tcomp.c).
  • Added test calls for __shfl, __shfl_up, __shfl_down, __shfl_xor, __ballot, __any, and __all in test_shfl.cu.

---

[Zaneham]

Hiya!

Welcome to Booth, and cheers for jumping in!

Thanks for finding that latent bug, and I verified it's fixed myself! Thank you so much.

A few things before it lands:

1. The test only compiles the kernel, it doesn't run it. Could you add an execution test that checks a real value? __ballot(pred) coming back as an actual lane mask instead of the mask constant is the clean one to pin down. There are harnesses in the tree for exactly this (the RDNA emulator, and nv_rt for NVIDIA).

2. It needs a rebase onto current master. You're on an older base (those 6 enc/tab failures are the base, not you!), and there's another PR in flight touching the same files, so rebasing keeps it tidy.

Minor one, now the lowerer always inserts the mask, those num_operands ternaries always take the same branch. Hardcoding index 1 like the AMD shuffle path does would read a bit clearer. Not blocking.

Kind regards,
Zane

[Zaneham]

Thanks for your contribution! Merging it now.