diff --git a/docs/source/kernel-library-selective-build.md b/docs/source/kernel-library-selective-build.md index f9a991767a3..3e9400f4d6a 100644 --- a/docs/source/kernel-library-selective-build.md +++ b/docs/source/kernel-library-selective-build.md @@ -36,17 +36,20 @@ The basic flow looks like this: ## APIs -We expose a CMake macro `[gen_selected_ops](https://github.com/pytorch/executorch/blob/main/tools/cmake/Codegen.cmake#L12)`, to allow users specifying op info: +We expose a CMake macro [gen_selected_ops](https://github.com/pytorch/executorch/blob/main/tools/cmake/Codegen.cmake#L12), to allow users specifying op info: ``` gen_selected_ops( - LIB_NAME # the name of the selective build operator library to be generated - OPS_SCHEMA_YAML # path to a yaml file containing operators to be selected - ROOT_OPS # comma separated operator names to be selected - INCLUDE_ALL_OPS # boolean flag to include all operators + LIB_NAME # the name of the selective build operator library to be generated + OPS_SCHEMA_YAML # path to a yaml file containing operators to be selected + ROOT_OPS # comma separated operator names to be selected + INCLUDE_ALL_OPS # boolean flag to include all operators + OPS_FROM_MODEL # path to a pte file of model to select operators from + DTYPE_SELECTIVE_BUILD # boolean flag to enable dtye selection ) ``` +The macro makes a call to gen_oplist.py, which requires a [distinct selection](https://github.com/BujSet/executorch/blob/main/codegen/tools/gen_oplist.py#L222-L228) of API choice. `OPS_SCHEMA_YAML`, `ROOT_OPS`, `INCLUDE_ALL_OPS`, and `OPS_FROM_MODEL` are mutually exclusive options, and should not be used in conjunction. ### Select all ops @@ -62,31 +65,29 @@ Context: each kernel library is designed to have a yaml file associated with it. This API lets users pass in a list of operator names. Note that this API can be combined with the API above and we will create a allowlist from the union of both API inputs. +### Select ops from model -## Example Walkthrough +This API lets users pass in a pte file of an exported model. When used, the pte file will be parsed to generate a yaml file that enumerates the operators and dtypes used in the model. -In CMakeLists.txt we have the following logic: -```cmake -set(_kernel_lib) -if(SELECT_ALL_OPS) - gen_selected_ops("" "" "${SELECT_ALL_OPS}") -elseif(SELECT_OPS_LIST) - gen_selected_ops("" "${SELECT_OPS_LIST}" "") -elseif(SELECT_OPS_YAML) - set(_custom_ops_yaml ${EXECUTORCH_ROOT}/examples/portable/custom_ops/custom_ops.yaml) - gen_selected_ops("${_custom_ops_yaml}" "" "") -endif() -``` -Then when calling CMake, we can do: +### Dtype Selective Build -``` -cmake -D… -DSELECT_OPS_LIST="aten::add.out,aten::mm.out” -``` +Beyond pruning the binary to remove unused operators, the binary size can further reduced by removing unused dtypes. For example, if your model only uses floats for the `add` operator, then including variants of the `add` operators for `doubles` and `ints` is unnecessary. The flag `DTYPE_SELECTIVE_BUILD` can be set to `ON` to support this additional optimization. Currently, dtype selective build is only supported with the model API described above. Once enabled, a header file that specifies only the operators and dtypes used by the model is created and linked against a rebuild of the `portable_kernels` lib. This feature is only supported for the portable kernels library; it's not supported for optimized, quantized or custom kernel libraries. -Or +## Example Walkthrough -``` -cmake -D… -DSELECT_OPS_YAML=ON -``` +In [CMakeLists.txt](https://github.com/BujSet/executorch/blob/main/examples/selective_build/CMakeLists.txt#L48-L72), we have the following cmake config options: + +1. `EXECUTORCH_SELECT_OPS_YAML` +2. `EXECUTORCH_SELECT_OPS_LIST` +3. `EXECUTORCH_SELECT_ALL_OPS` +4. `EXECUTORCH_SELECT_OPS_FROM_MODEL` +5. `EXECUTORCH_DTYPE_SELECTIVE_BUILD` + +These options allow a user to tailor the cmake build process to utilize the different APIs, and results in different invocations on the `gen_selected_ops` [function](https://github.com/BujSet/executorch/blob/main/examples/selective_build/CMakeLists.txt#L110-L123). The following table describes some examples of how the invocation changes when these configs are set: -To select from either an operator name list or a schema yaml from kernel library. +| Example cmake Call | Resultant `gen_selected_ops` Invocation | +| :----: | :---:| +|
cmake -D… -DSELECT_OPS_LIST="aten::add.out,aten::mm.out"
|
gen_selected_ops("" "${SELECT_OPS_LIST}" "" "" "")
| +|
cmake -D… -DSELECT_OPS_YAML=ON
|
set(_custom_ops_yaml ${EXECUTORCH_ROOT}/examples/portable/custom_ops/custom_ops.yaml)
gen_selected_ops("${_custom_ops_yaml}" "" "")
| +|
cmake -D… -DEXECUTORCH_SELECT_OPS_FROM_MODEL="model.pte.out"
|
gen_selected_ops("" "" "" "${_model_path}" "")
| +|
cmake -D… -DEXECUTORCH_SELECT_OPS_FROM_MODEL="model.pte.out" -DEXECUTORCH_DTYPE_SELECTIVE_BUILD=ON
|
gen_selected_ops("" "" "" "${_model_path}" "ON")
| diff --git a/examples/selective_build/README.md b/examples/selective_build/README.md index 97706d70c48..888020ce770 100644 --- a/examples/selective_build/README.md +++ b/examples/selective_build/README.md @@ -12,12 +12,12 @@ cd executorch bash examples/selective_build/test_selective_build.sh cmake ``` -Check out `CMakeLists.txt` for demo of 3 selective build APIs: +Check out `CMakeLists.txt` for demo of selective build APIs: 1. `SELECT_ALL_OPS`: Select all ops from the dependency kernel libraries, register all of them into ExecuTorch runtime. 2. `SELECT_OPS_LIST`: Only select operators from a list. 3. `SELECT_OPS_YAML`: Only select operators from a yaml file. +4. `SELECT_OPS_FROM_MODEL`: Only select operators from a from an exported model pte. +5. `DTYPE_SELECTIVE_BUILD`: Enable rebuild of `portable_kernels` to use dtype selection. Currently only supported for `SELECTED_OPS_FROM_MODEL` API and `portable_kernels` lib. Other configs: - `MAX_KERNEL_NUM=N`: Only allocate memory for N operators. - -We have one more API incoming: only select from an exported model file (.pte).