ROCm hip-ep

hip-ep is an ONNX Runtime Execution Provider for AMD GPUs. It compiles ONNX graphs through an MLIR pipeline—from ONNX dialect operations to a custom HIP dialect and LLVM IR—and executes them with hipDNN, MIOpen, hipBLASLt, and custom HIP kernels.

The provider integrates with ONNX Runtime through the MorphiZen pass framework. By default, compiled models are emitted as OS-portable LLVM bitcode and JIT-loaded in-process when a session is created. Native per-model libraries remain available as an opt-in artifact format.

Highlights

• MLIR compiler pipeline — lowers ONNX graphs to HIP and LLVM IR.
• ROCm execution backends — dispatches to hipDNN, MIOpen, hipBLASLt, and custom HIP kernels.
• Dynamic shapes — supports runtime batch/sequence dimensions, shape refinement, and runtime-sized outputs.
• GPU memory planning — packs transient allocations into one or more grow-on-demand pool domains and keeps host-written shape scalars in separate host-mapped scratch.
• In-graph output allocation — allocates graph outputs through the Execution Provider callback once their runtime shapes are known.
• Externalized constants — stores large model weights in a sidecar constants file instead of embedding them in the model artifact.
• Two artifact formats — LLVM bitcode with in-process JIT by default, or a native .dll/.so when explicitly requested.
• Extensible compiler pipeline — supports registered plugin slots and out-of-tree dialect/pass plugins.

Get started

Goal	Guide
Build and run on Windows	Windows quick start
Build on Linux, use Docker, or download a prebuilt package	Linux quick start
Understand artifact formats	LLVM IR vs native artifacts
Extend the pipeline with a plugin	Plugin authoring guide
Contribute to the project	Contributing guide

How it works

ONNX model
    │
    ▼
ONNX Runtime + MorphiZen EP integration
    │
    ▼
ONNX MLIR
    │  ONNX-to-HIP lowering
    ▼
HIP MLIR
    │  shape refinement, bufferization, and memory planning
    ▼
LLVM dialect
    │  HIP-to-LLVM lowering + generated model interface
    ▼
Per-model artifact + constants file
    ├── LLVM bitcode (default) ──► in-process ORC JIT
    └── native .dll/.so (opt-in) ─► OS dynamic loader
    │
    ▼
hipDNN / MIOpen / hipBLASLt / custom HIP kernels
    │
    ▼
AMD GPU

The generated model interface initializes per-session runtime state, executes the compiled graph, and releases resources. Graph outputs are allocated in-graph through the provider's output-allocation callback; transient buffers use lifetime-aware GPU pool planning.

For the exact registered pass names and ordering, see the pipeline pass menu. For compiler/runtime ABI details, see the compiler-runtime contract.

Supported operations

hip-ep supports common operation families used by transformer, vision, convolutional, quantized, and multimodal models:

• convolution, transposed convolution, and pooling;
• MatMul, Gemm, quantized MatMul, and mixture-of-experts operations;
• attention, grouped-query attention, rotary embeddings, and KV-cache updates;
• normalization and activation functions;
• elementwise, reduction, gather/scatter, padding, resize, and indexing operations;
• tensor shape/view operations and selected ONNX control flow.

See Supported operations for per-operation domains, lowering paths, runtime backends, data types, dynamic-shape behavior, and known limitations.

Developer tools

Tool	Purpose
hip-mlir-opt	Run and inspect registered HIP/MLIR passes and pipelines.
hip-compiler	Compile ONNX MLIR into LLVM-bitcode or native model artifacts.
hip-onnx-runner	Run ONNX models through the installed Execution Provider.
hip-inspect	Inspect metadata embedded in a compiled model artifact.
hip-test	Load and execute compiled artifacts through the same runtime path used by the EP.

Tool availability depends on the build configuration; see the platform quick-start guide for the corresponding CMake options and install layout.

Documentation

Architecture and compiler

• Pipeline pass menu
• Compiler-runtime contract
• Shape inference design
• GPU memory planning
• Output allocator design
• Constant handling
• Compilation options
• LLVM IR vs native artifacts

Extending and operating

• Plugin authoring
• Plugin interface design
• Custom kernel design
• Per-operation profiling
• Remote development workflow

Developer entry points

• ONNX frontend: lib/Conversion/OnnxToHip/
• HIP dialect: include/hip/Dialect/ and lib/Dialect/
• HIP-to-LLVM lowering: lib/Conversion/HipToLLVM/
• Compiler driver: lib/Compiler/
• Runtime and custom kernels: lib/Runtime/
• MorphiZen/ONNX Runtime integration: backend-mlir-compiler/ and morphizen/
• Command-line tools: tools/
• Tests: test/lit/, test/numeric/, test/python/, and test/e2e/

Issues and feedback

Report bugs and request features through GitHub Issues.

Contributing

This project follows LLVM's incremental development and AI tool use policies. See CONTRIBUTING.md for project-specific requirements.

License

Copyright (C) 2026 Advanced Micro Devices, Inc. All rights reserved.

Licensed under the MIT License. See LICENSE.