AIE2 vector ISA: is there runtime-indexed gather/scatter? (aie::shuffle is structured-only → blocks SIMD stream-compaction)

[Scottcjn]

Context. I've been hand-authoring AIE2 kernels (gen-1 XDNA1 / Phoenix, npu1, via IRON + Peano) for a non-bijunctive "collapse/prune" primitive: threshold a vector, then left-pack the survivors into a dense prefix (stream compaction). Repo for context: https://github.com/Scottcjn/open-xdna

What I found in the vector API:

• aie::shuffle and friends (shuffle_up/shuffle_down, rotate, interleave, select) are structured — the permute/select pattern is fixed at compile time or a fixed shift, not a data-dependent runtime index vector.
• I couldn't find a runtime-indexed gather/scatter — i.e. dst[i] = src[idx[i]] / dst[idx[i]] = src[i] where idx is a runtime vector — analogous to x86 vpgatherdd/pshufb or PPC vec_perm with a runtime control vector. Nor a load_gather/store_scatter.

Consequence. SIMD stream-compaction (left-pack) isn't directly expressible on the AIE2 vector unit, because the placement step needs a data-dependent scatter. I vectorized the threshold/select fine, but fell back to the scalar unit for the actual compaction.

Questions:

1. Is there a runtime-indexed gather/scatter, or a dynamic (runtime control vector) permute, in the AIE2 vector ISA / aie API that I missed?
2. Is there an idiomatic vectorized stream-compaction pattern in IRON (e.g. a prefix-sum + scatter idiom) you'd recommend — or is scalar-unit compaction the expected approach today?
3. Is data-dependent gather/scatter on the roadmap for future AIE generations?

Not a complaint — just trying to confirm whether this is a genuine ISA boundary or a missing API. Thanks!

[fifield]

Questions:

1. Is there a runtime-indexed gather/scatter, or a dynamic (runtime control vector) permute, in the AIE2 vector ISA / aie API that I missed?

No. There is no runtime-indexed gather, scatter, or arbitrary-permute instruction in the AIE2 ISA.

2. Is there an idiomatic vectorized stream-compaction pattern in IRON (e.g. a prefix-sum + scatter idiom) you'd recommend — or is scalar-unit compaction the expected approach today?

I don't know of an existing example, so I've attempted a vectorized butterfly network to implement the left-pack step, which beats the naive scalar version by ~3.6x: https://github.com/Xilinx/mlir-aie/tree/vector_compact/programming_examples/basic/vector_compact I've only tested on aie2p.