MLIR-based AI Engine toolchain

This repository contains an MLIR-based toolchain for AI Engine-enabled devices, such as AMD Ryzen™ AI and Versal™. This repository can be used to generate low-level configurations for the AI Engine portion of these devices. AI Engines are organized as a spatial array of tiles, where each tile contains AI Engine cores and/or memories. The spatial array is connected by stream switches that can be configured to route data between AI Engine tiles scheduled by their programmable Data Movement Accelerators (DMAs). This repository contains MLIR representations, with multiple levels of abstraction, to target AI Engine devices. This enables compilers and developers to program AI Engine cores, as well as describe data movements and array connectivity. A Python API is made available as a convenient interface for generating MLIR design descriptions. Backend code generation is also included, targeting the aie-rt library.

This project is primarily intended to support the open-source community, particularly tool builders, with low-level access to AIE devices and enable the development of a wide variety of programming models from higher level abstractions. We provide an example programming flow: Interface Representation for hands-ON (IRON) close-to-metal programming of the AIE-array. IRON is an open access toolkit enabling performance engineers to build fast and efficient, often specialized designs through a set of Python language bindings around the mlir-aie dialect. As such, it contains some examples, however this project is not intended to represent an end-to-end compilation flow for all application designs. If you're looking for an out-of-the-box experience for highly efficient machine learning, check out the AMD Ryzen™ AI Software Platform.

Getting Started for AMD Ryzen™ AI on Linux

These instructions will guide you through everything required for building and executing a program on the Ryzen™ AI NPU, starting from a fresh bare-bones Ubuntu 24.04 or Ubuntu 24.10 install.

Initial Setup

Be sure you have the latest BIOS on your laptop or mini-PC that enables the NPU. See here.

If starting from Ubuntu 24.04 you may need to update the Linux kernel to 6.11+ by installing the Hardware Enablement (HWE) stack:

sudo apt update 
sudo apt install --install-recommends linux-generic-hwe-24.04
sudo reboot

Prerequisites

BIOS Settings:

Turn off SecureBoot (Allows for unsigned drivers to be installed): BIOS → Security → Secure boot → Disable

Build and install the XDNA™ Driver and XRT

1. Execute the scripted build process:

   This script will install package dependencies, build the xdna-driver and xrt packages, and install them. These steps require sudo access.

   bash ./utils/build_drivers.sh

2. Reboot as directed after the script exits.

   sudo reboot

3. Check that the NPU is working if the device appears with xrt-smi:

   source /opt/xilinx/xrt/setup.sh
   xrt-smi examine

   At the bottom of the output you should see:

   Devices present
   BDF             :  Name             
   ------------------------------------
   [0000:66:00.1]  :  RyzenAI-npu1
   

Install IRON and MLIR-AIE Prerequisites

1. Install the following packages needed for MLIR-AIE:

   sudo apt install \
   build-essential clang clang-14 lld lld-14 cmake python3-venv python3-pip

2. Install opencv which is needed for some programming examples:

   sudo apt install libopencv-dev python3-opencv

Install IRON for AMD Ryzen™ AI AIE Application Development

1. Clone the mlir-aie repository:

   git clone https://github.com/Xilinx/mlir-aie.git
   cd mlir-aie

2. Install IRON library, mlir-aie and llvm-aie compilers from whls:

   source utils/quick_setup.sh

Build an IRON Design for AIEs in the AMD Ryzen™ AI NPU

For your design of interest, for instance from programming_examples, 2 steps are needed: (i) build the AIE design and then (ii) build the host code.

Build Device AIE Part

1. Goto the design of interest and run:

   make

2. Build host code and execute the design:

   make run

Learn more about NPU programming with IRON

1. Continue to the IRON AIE Application Programming Guide

2. Some MLIR-AIE documentation is available on the website

Optional: Install AIETools

You may skip the Vitis™ installation step if you intend to only target AMD XDNA™/AIE-ML (AIE2) and AMD XDNA™ 2 (AIE2P) using our open-source single-core compiler Peano. Compiling with xchesscc is not supported without installing AMD Vitis™ AIE Essentials.

1. Install Vitis™ AIE Essentials from Ryzen AI Software 1.3 Early Accesss. We will assume you use the installation directory, /tools/ryzen_ai-1.3.0/vitis_aie_essentials.

   This is an early access lounge, you must register and be granted access at this time.

   1. Download VAIML Installer for Linux based compilation: ryzen_ai-1.3.0ea1.tgz

   2. Extract the required tools:

         tar -xzvf ryzen_ai-1.3.0ea1.tgz
         cd ryzen_ai-1.3.0
         mkdir vitis_aie_essentials
         mv vitis_aie_essentials*.whl vitis_aie_essentials
         cd vitis_aie_essentials
         unzip vitis_aie_essentials*.whl

2. Set up an AI Engine license.

   1. Get a local license for AI Engine tools from https://www.xilinx.com/getlicense.

   2. Copy your license file (Xilinx.lic) to your preferred location, e.g. /opt/Xilinx.lic:

3. Setup your environment using the following script for Vitis™ for AIETools:

   #!/bin/bash
    #################################################################################
    # Setup Vitis AIE Essentials
    #################################################################################
    export AIETOOLS_ROOT=/tools/ryzen_ai-1.3.0/vitis_aie_essentials
    export PATH=$PATH:${AIETOOLS_ROOT}/bin
    export LM_LICENSE_FILE=/opt/Xilinx.lic

Troubleshooting:

Update BIOS:

Be sure you have the latest BIOS for your laptop or mini PC, this will ensure the NPU (sometimes referred to as IPU) is enabled in the system. You may need to manually enable the NPU: Advanced → CPU Configuration → IPU

NOTE: Some manufacturers only provide Windows executables to update the BIOS, please do this before installing Ubuntu.

Detailed Getting Started Guides and Documentation:

Getting Started on a Versal™ board

Running on a Versal™ board

Getting Started and Running on Windows Ryzen™ AI

Getting Started and Running on Linux Ryzen™ AI

IRON AIE Application Programming Guide

MLIR Dialect and Compiler Documentation

Interested in contributing MLIR-AIE? Information for developers

---

Copyright© 2019-2024 Advanced Micro Devices, Inc