Skip to content

Commit

Permalink
Fix docs for multiple accelerators (#2804)
Browse files Browse the repository at this point in the history
Signed-off-by: Andrea Somaini <andrea@somaini.dev>
  • Loading branch information
thegoldgoat committed Apr 23, 2024
1 parent ee7eaca commit 4400cbc
Showing 1 changed file with 26 additions and 4 deletions.
30 changes: 26 additions & 4 deletions docs/AddCustomAccelerators.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ In general, onnx-mlir handles custom accelerators as pluggins which can be turne
Besides this document, [NNPA accelerator](../src/Accelerators/NNPA) can be used as an example that has been deployed in onnx-mlir.

## 1. Code folder

In onnx-mlir, all code for an accelerator should be put inside a separate folder under `src/Accelerators`. Thus, the first step to support an accelerator is to create a folder for it inside `src/Accelerators`.

The folder name will be used as the accelerator name in onnx-mlir. In particular, it is used to
Expand All @@ -15,22 +16,25 @@ The folder name will be used as the accelerator name in onnx-mlir. In particular
The folder content is flexible depending on each accelerator. However, we recomment to follow the same structure as the root folder of `onnx-mlir` as much as possbile. This helps maintain the consitency across the whole project.

### 1.1 Build accelerators in onnx-mlir
To build accelerators in onnx-mlir, use the cmake variable `ONNX_MLIR_ACCELERATORS` when building onnx-mlir. `ONNX_MLIR_ACCELERATORS` accepts a comma-separated list of accelerator names. For example,

To build accelerators in onnx-mlir, use the cmake variable `ONNX_MLIR_ACCELERATORS` when building onnx-mlir. `ONNX_MLIR_ACCELERATORS` accepts a semicolon-separated list of accelerator names. For example,
```bash
$ cd build
$ cmake .. -DONNX_MLIR_ACCELERATORS=accel1,accel2
$ cmake .. -DONNX_MLIR_ACCELERATORS=accel1;accel2
```

### 1.2 Compile a model to run with selected accelerators.
The compiler command `onnx-mlir` has an option, i.e. `--maccel`, to compile a model for selected accelerators. `--maccel` accepts a comma-separated list of accelerator names. For example,

The compiler command `onnx-mlir` has an option, i.e. `--maccel`, to compile a model for selected accelerators. For each accelerator add a `--maccel=accel_name` entry. For example,

```bash
$ onnx-mlir --maccel=accel1,accel2 model.onnx
$ onnx-mlir --maccel=accel1 --maccel=accel2 model.onnx
```

Only built accelerators can be used with `--maccel`.

### 1.3 Run passes related to selected accelerators.

Passes defined by an accelerator can be run or tested via `onnx-mlir-opt` command by using option `--maccel` which is similar to `--maccel` in `onnx-mlir` (See Sec. [1.2](#1.2-compile-a-model-to-run-with-selected-accelerators)). For example, to call a pass `--optimize-data-layout` defined by accelerator `accel1`:

```bash
Expand All @@ -41,6 +45,24 @@ Only built accelerators can be used with `--maccel`.

## 2. Code integration

### 2.1 Macro

Each accelerator is required to define a few macros. These needs to be included in [onnx_mlir::accel::Accelerator](../src/Accelerators/Accelerator.hpp). These macros are:

1. `INSTRUMENTSTAGE_ENUM_<accel_name>`
2. `INSTRUMENTSTAGE_CL_ENUM_<accel_name>`
3. `PROFILEIR_CL_ENUM_<accel_name>`

Replace `<accel_name>` with the name of the accelerator, for example if your accelerator is named `ACCEL1` use:

```C
#define INSTRUMENTSTAGE_ENUM_ACCEL1
#define INSTRUMENTSTAGE_CL_ENUM_ACCEL1
#define PROFILEIR_CL_ENUM_ACCEL1
```

### 2.2 Dialects and passes

Writing code in MLIR typically involves desiging dialects and passes. So does supporting an accelerator. Thus, to integrate accelerator code into onnx-mlir is to register dialects and passes in onnx-mlir.

We provide a base class [onnx_mlir::accel::Accelerator](../src/Accelerators/Accelerator.hpp) from which users can define an inherited class and write hooks to register dialects and passes.
Expand Down

0 comments on commit 4400cbc

Please sign in to comment.