Update selective build example + CI for top-level targets

examples/selective_build/README.md

@@ -1,23 +1,137 @@
 # Selective Build Examples
-To optimize binary size of ExecuTorch runtime, selective build can be used. This folder contains examples to select only the operators needed for ExecuTorch build. This example will demonstrate the CMake build.
+To optimize binary size of ExecuTorch runtime, selective build can be used. This folder contains examples to select only the operators needed for ExecuTorch build.
 
-## How to run
+These examples showcase two flows - the simple way, using CMake options to configure the framework build, and an advanced flow - showcasing user-defined kernel targets including custom operators.
 
 Prerequisite: finish the [setting up wiki](https://pytorch.org/executorch/main/getting-started-setup).
 
-Run:
+## Example 1 - Basic Flow
 
-```bash
-cd executorch
-bash examples/selective_build/test_selective_build.sh cmake
+This example showcases using CMake options to control which operators are included. This approach should be preferred when not using
+custom operators or additional kernel libraries beyond the standard kernels provided by ExecuTorch.
+
+The code under the basic/ directory builds a minimal model runner binary which links to a selective kernel target. To build the
+example with operators needed for the MobileNetV2 model, run the following commands:
+```
+# From the executorch directory
+python -m examples.portable.scripts.export --model_name="mv2" # Create a PTE file for MobileNetV2
+cd examples/selective_build/basic
+mkdir cmake-out && cd cmake-out
+cmake .. -DEXECUTORCH_SELECT_OPS_MODEL="../../mv2.pte" # Build with kernels needed for mv2.pte
+cmake --build . -j8
+./selective_build_test --model_path="../../mv2.pte" # Run the model with the selective kernel library
+```
+
+### CMake Options
+
+The example commands above show use of the EXECUTORCH_SELECT_OPS_MODEL option to select operators used in a PTE file, but there are
+several ways to provide the operator list. The options can be passed to CMake in the same way (during configuration) and are mutually
+exclusive, meaning that only one of these options should be chosen.
+
+ * `EXECUTORCH_SELECT_OPS_MODEL`: Select operators used in a .PTE file. Takes a path to the file.
+ * `EXECUTORCH_SELECT_OPS_YAML`: Provide a list of operators from a .yml file, typically generated with the `codegen/tools/gen_oplist.py` script. See this script for usage information.
+ * `EXECUTORCH_SELECT_OPS_LIST`: Provide a comma-separated list of operators to include. An example is included below.
+
+Example operator list specification (passed as a CLI arg to the CMake configure command):
+```
+-DEXECUTORCH_SELECT_OPS_LIST="aten::convolution.out,\
+aten::_native_batch_norm_legit_no_training.out,aten::hardtanh.out,aten::add.out,\
+aten::mean.out,aten::view_copy.out,aten::permute_copy.out,aten::addmm.out,\
+aten,aten::clone.out"
+```
+
+#### DType-Selective Build
+
+To further reduce binary size, ExecuTorch can specialize the individual operators for only the dtypes (data types) used. For example, if
+the model only calls add with 32-bit floating point tensors, it can drop parts of the code that handle integer tensors or other floating point types. This option is controlled by passing `-DEXECUTORCH_DTYPE_SELECTIVE_BUILD=ON` to CMake. It is only supported in conjunction
+with the `EXECUTORCH_SELECT_OPS_MODEL` option and is not yet supported for other modes. It is recommended to enable this option when using `EXECUTORCH_SELECT_OPS_MODEL` as it provides significant size savings on top of the kernel selective build.
+
+### How it Works
+
+The CMake options described above are read by ExecuTorch framework build, which is referenced via `add_subdirectory` in basic/CMakeLists.txt. These options reflect in the `executorch_kernels` CMake target, which is linked against the example binary.
+
+```cmake
+# basic/CMakeLists.txt
+target_link_libraries(
+  selective_build_test
+  PRIVATE executorch_core extension_evalue_util extension_runner_util
+          gflags::gflags executorch_kernels
+)
+```
+
+To use selective build in a user CMake project, take the following steps:
+ * Reference the executorch framework via `add_subdirectory`.
+ * Add `executorch_kernels` as a dependency (via target_link_libraries).
+ * Set CMake options at build time or in user CMake code.
+
+To use the CMake-build framework libraries from outside of the CMake ecosystem, link against libexecutorch_selected_kernels.
+
+## Example 2 - Advanced Flow for Custom Ops and Kernel Libraries
+
+This example showcases defined a custom kernel target. This option can be used when defining custom operators or integrating with
+kernel libraries not part of the standard ExecuTorch build.
+
+The code under the advanced/ directory builds a minimal model runner binary which links to a user-defined kernel library target. To run a model with a simple custom operator, run the following commands:
+```
+# From the executorch directory
+python -m examples.portable.custom_ops.custom_ops_1 # Create a model PTE file
+cd examples/selective_build/basic
+mkdir cmake-out && cd cmake-out
+cmake .. -DEXECUTORCH_SELECT_OPS_MODEL="../../custom_ops_1.pte" -DEXECUTORCH_EXAMPLE_USE_CUSTOM_OPS=ON # Build with kernels needed for the model
+cmake --build . -j8
+./selective_build_test --model_path="../../custom_ops_1.pte" # Run the model with the selective kernel library
+```
+
+### CMake Options
+
+The CMake logic in `advanced/CMakeLists.txt` respects the CMake options described in the basic flow, as well as the following options:
+
+ * `EXECUTORCH_EXAMPLE_USE_CUSTOM_OPS`: Build and link some simple custom operators.
+ * `EXECUTORCH_EXAMPLE_SELECT_ALL_OPS`: Build a kernel target with all available operators.
+
+### How it Works
+
+The build logic in `advanced/CMakeLists.txt` uses the `gen_selected_ops`, `generate_bindings_for_kernels`, and `gen_operators_lib` CMake functions to define an operator target. See [Kernel Library Selective Build](https://docs.pytorch.org/executorch/main/kernel-library-selective-build.html) for more information on selective build.
+
+```cmake
+gen_selected_ops(
+  LIB_NAME
+  "select_build_lib"
+  OPS_SCHEMA_YAML
+  "${_custom_ops_yaml}"
+  ROOT_OPS
+  "${EXECUTORCH_SELECT_OPS_LIST}"
+  INCLUDE_ALL_OPS
+  "${EXECUTORCH_SELECT_ALL_OPS}"
+  OPS_FROM_MODEL
+  "${EXECUTORCH_SELECT_OPS_MODEL}"
+  DTYPE_SELECTIVE_BUILD
+  "${EXECUTORCH_DTYPE_SELECTIVE_BUILD}"
+)
+
+generate_bindings_for_kernels(
+  LIB_NAME
+  "select_build_lib"
+  FUNCTIONS_YAML
+  ${EXECUTORCH_ROOT}/kernels/portable/functions.yaml
+  CUSTOM_OPS_YAML
+  "${_custom_ops_yaml}"
+  DTYPE_SELECTIVE_BUILD
+  "${EXECUTORCH_DTYPE_SELECTIVE_BUILD}"
+)
+
+gen_operators_lib(
+  LIB_NAME
+  "select_build_lib"
+  KERNEL_LIBS
+  ${_kernel_lib}
+  DEPS
+  executorch_core
+  DTYPE_SELECTIVE_BUILD
+  "${EXECUTORCH_DTYPE_SELECTIVE_BUILD}"
+)
 ```
 
-Check out `CMakeLists.txt` for demo of selective build APIs:
-1. `SELECT_ALL_OPS`: Select all ops from the dependency kernel libraries, register all of them into ExecuTorch runtime.
-2. `SELECT_OPS_LIST`: Only select operators from a list.
-3. `SELECT_OPS_YAML`: Only select operators from a yaml file.
-4. `SELECT_OPS_FROM_MODEL`: Only select operators from a from an exported model pte.
-5. `DTYPE_SELECTIVE_BUILD`: Enable rebuild of `portable_kernels` to use dtype selection. Currently only supported for `SELECTED_OPS_FROM_MODEL` API and `portable_kernels` lib.
+To link against this target, the top-level binary target declares a dependency on `select_build_lib`, which is the library name defined by the above function invocations. To use outside of the CMake ecosystem, link against libselect_build_lib.
 
-Other configs:
-- `MAX_KERNEL_NUM=N`: Only allocate memory for N operators.
+See `test_selective_build.sh` for additional build examples.

examples/selective_build/CMakeLists.txt → examples/selective_build/advanced/CMakeLists.txt

@@ -18,7 +18,7 @@
 cmake_minimum_required(VERSION 3.19)
 project(selective_build_example)
 
-set(EXECUTORCH_ROOT ${CMAKE_CURRENT_SOURCE_DIR}/../..)
+set(EXECUTORCH_ROOT ${CMAKE_CURRENT_SOURCE_DIR}/../../..)
 set(TORCH_ROOT ${EXECUTORCH_ROOT}/third-party/pytorch)
 
 include(${EXECUTORCH_ROOT}/tools/cmake/Utils.cmake)
@@ -37,56 +37,37 @@ set(_common_compile_options -Wno-deprecated-declarations -fPIC
                             -ffunction-sections -fdata-sections
 )
 
-# Let files say "include <executorch/path/to/header.h>".
-set(_common_include_directories ${EXECUTORCH_ROOT}/..)
-
-find_package(executorch CONFIG REQUIRED)
-find_package(
-  gflags REQUIRED PATHS ${CMAKE_CURRENT_BINARY_DIR}/../../third-party
-)
-
-target_include_directories(
-  executorch_core INTERFACE ${_common_include_directories}
-)
+add_subdirectory(${EXECUTORCH_ROOT} ${CMAKE_CURRENT_BINARY_DIR}/executorch)
 
 # ------------------------------ OPTIONS BEGIN -------------------------------
 
-# Option to register ops from yaml file
-option(EXECUTORCH_SELECT_OPS_YAML "Register all the ops from a given yaml file"
-       OFF
-)
-
-# Option to register op list
-option(EXECUTORCH_SELECT_OPS_LIST "Register a list of ops, separated by comma"
-       OFF
-)
-
 # Selective build options.
-option(EXECUTORCH_SELECT_ALL_OPS
+option(EXECUTORCH_EXAMPLE_SELECT_ALL_OPS
        "Whether to register all ops defined in portable kernel library." OFF
 )
 
-# Option to enable parsing ops and dtypes directly from model pte file
-option(EXECUTORCH_SELECT_OPS_FROM_MODEL
-       "Enable op selection from pte during build." OFF
+option(EXECUTORCH_EXAMPLE_USE_CUSTOM_OPS
+       "Whether to include custom ops in the example." OFF
 )
 
-# Option to enable dtype selective build. Note: must be using selective build
-# model API.
-option(EXECUTORCH_DTYPE_SELECTIVE_BUILD "Enable dtype selective build." OFF)
+# Note that the following options are defined by the core framework and are also
+# used by this example when defining a custom operator target:
+#
+# EXECUTORCH_SELECT_OPS_YAML EXECUTORCH_SELECT_OPS_LIST
+# EXECUTORCH_SELECT_OPS_MODEL EXECUTORCH_DTYPE_SELECTIVE_BUILD
+
 # ------------------------------- OPTIONS END --------------------------------
 
 #
 # The `_<target>_srcs` lists are defined by executorch_load_build_variables.
 #
 executorch_load_build_variables()
 
-#
-# select_build_lib: C++ library to register selected ops in custom kernel
-# library
-#
+# For advanced use cases, we can define a custom operator target. This is useful
+# when using custom operators.
 set(_kernel_lib)
-if(EXECUTORCH_SELECT_OPS_YAML)
+
+if(EXECUTORCH_EXAMPLE_USE_CUSTOM_OPS)
   set(_custom_ops_yaml
       ${EXECUTORCH_ROOT}/examples/portable/custom_ops/custom_ops.yaml
   )
@@ -116,7 +97,7 @@ gen_selected_ops(
   INCLUDE_ALL_OPS
   "${EXECUTORCH_SELECT_ALL_OPS}"
   OPS_FROM_MODEL
-  "${EXECUTORCH_SELECT_OPS_FROM_MODEL}"
+  "${EXECUTORCH_SELECT_OPS_MODEL}"
   DTYPE_SELECTIVE_BUILD
   "${EXECUTORCH_DTYPE_SELECTIVE_BUILD}"
 )
@@ -143,6 +124,9 @@ gen_operators_lib(
   "${EXECUTORCH_DTYPE_SELECTIVE_BUILD}"
 )
 
+executorch_target_link_options_shared_lib(select_build_lib)
+set(selected_kernel_target select_build_lib)
+
 list(TRANSFORM _executor_runner__srcs PREPEND "${EXECUTORCH_ROOT}/")
 
 #
@@ -154,8 +138,8 @@ if(NOT CMAKE_BUILD_TYPE STREQUAL "Debug")
   target_link_options_gc_sections(selective_build_test)
 endif()
 target_link_libraries(
-  selective_build_test PRIVATE executorch_core extension_evalue_util
-                               extension_runner_util gflags select_build_lib
+  selective_build_test
+  PRIVATE executorch_core extension_evalue_util extension_runner_util
+          gflags::gflags ${selected_kernel_target}
 )
-executorch_target_link_options_shared_lib(select_build_lib)
 target_compile_options(selective_build_test PUBLIC ${_common_compile_options})

examples/selective_build/basic/CMakeLists.txt

@@ -0,0 +1,77 @@
+# Copyright (c) Meta Platforms, Inc. and affiliates.
+# All rights reserved.
+#
+# This source code is licensed under the BSD-style license found in the
+# LICENSE file in the root directory of this source tree.
+
+#
+# Simple CMake build system for selective build demo.
+#
+# ### Editing this file ###
+#
+# This file should be formatted with
+# ~~~
+# cmake-format -i CMakeLists.txt
+# ~~~
+# It should also be cmake-lint clean.
+#
+cmake_minimum_required(VERSION 3.19)
+project(selective_build_example)
+
+set(EXECUTORCH_ROOT ${CMAKE_CURRENT_SOURCE_DIR}/../../..)
+set(TORCH_ROOT ${EXECUTORCH_ROOT}/third-party/pytorch)
+
+include(${EXECUTORCH_ROOT}/tools/cmake/Utils.cmake)
+include(${EXECUTORCH_ROOT}/tools/cmake/Codegen.cmake)
+
+if(NOT PYTHON_EXECUTABLE)
+  resolve_python_executable()
+endif()
+
+if(NOT CMAKE_CXX_STANDARD)
+  set(CMAKE_CXX_STANDARD 17)
+  # Can't set to 11 due to executor_runner.cpp make_unique
+endif()
+
+set(_common_compile_options -Wno-deprecated-declarations -fPIC
+                            -ffunction-sections -fdata-sections
+)
+
+add_subdirectory(${EXECUTORCH_ROOT} ${CMAKE_CURRENT_BINARY_DIR}/executorch)
+
+# ------------------------------ OPTIONS BEGIN -------------------------------
+
+# The following options are defined by the core framework and are also used in
+# the generated kernel target.
+#
+# EXECUTORCH_SELECT_OPS_YAML EXECUTORCH_SELECT_OPS_LIST
+# EXECUTORCH_SELECT_OPS_MODEL EXECUTORCH_DTYPE_SELECTIVE_BUILD
+
+# ------------------------------- OPTIONS END --------------------------------
+
+#
+# The `_<target>_srcs` lists are defined by executorch_load_build_variables.
+#
+executorch_load_build_variables()
+
+# For most use cases, we can configure the ExecuTorch kernel library build using
+# the EXECUTORCH_SELECT_OPS_* variables. This will reflect in the
+# executorch_kernels target, which includes the configured kernel libraries,
+# including selective build, where supported.
+
+list(TRANSFORM _executor_runner__srcs PREPEND "${EXECUTORCH_ROOT}/")
+
+#
+# selective_build_test: test binary to allow different operator libraries to
+# link to
+#
+add_executable(selective_build_test ${_executor_runner__srcs})
+if(NOT CMAKE_BUILD_TYPE STREQUAL "Debug")
+  target_link_options_gc_sections(selective_build_test)
+endif()
+target_link_libraries(
+  selective_build_test
+  PRIVATE executorch_core extension_evalue_util extension_runner_util
+          gflags::gflags executorch_kernels
+)
+target_compile_options(selective_build_test PUBLIC ${_common_compile_options})