[feature] [MIOpen] Exploring MIOpen Fusion API integration opportunities

[MarijaGijic]

I want to explore adding support for fused pointwise operations (e.g., fusing sequential chains of elementwise math like Add ➔ Mul ➔ ReLU or Scale ➔ TensorOp).
Since the native MIOpen Fusion API is designed for heavy anchors (like Convolution or BatchNorm) and does not support standalone, pure pointwise math chains (like arbitrary sequences of Add ➔ Mul ➔ ReLU), I want to ask:
What is the preferred engineering approach to implement pure pointwise operator fusion within hipDNN?
Issue: #9016

[averinevg]

For implementing a pointwise op sequence, a few options depending on whether you need a single fused kernel:

1. Hand-written fused HIP kernel in hip-kernel-provider. For the whole chain in one kernel, the most direct route is to add a HIP kernel under dnn-providers/hip-kernel-provider — there's existing kernel/engine machinery there to model on.

2. The rocke engine (also in hip-kernel-provider). It has a kernel builder and composable building blocks, which is close to what you want. Caveat: it's relatively new and still early-stage, and I'm not working on it directly, so treat it as a promising direction to study rather than turnkey.

3. Compose from existing MIOpen ops (if a single fused kernel isn't required). miopenOpTensor computes C = op(alpha1 * A, alpha2 * B) + beta * C for op in {Add, Mul, Min, Max}, and miopenActivationForward handles ReLU and others — so Add → Mul → ReLU maps to a short sequence of existing calls. Not a fused kernel (multiple launches + intermediates), but the lowest-effort path to a working result, e.g. as a new engine that orchestrates the ops.