diff --git a/backends/apple/mps/setup.md b/backends/apple/mps/setup.md index c8fdfeb98e4..697d93ea659 100644 --- a/backends/apple/mps/setup.md +++ b/backends/apple/mps/setup.md @@ -111,7 +111,7 @@ python3 -m examples.apple.mps.scripts.mps_example --model_name="mv3" --no-use_fp ``` ### Profiling: -1. [Optional] Generate an [ETRecord](./sdk-etrecord.rst) while you're exporting your model. +1. [Optional] Generate an [ETRecord](./etrecord.rst) while you're exporting your model. ```bash cd executorch python3 -m examples.apple.mps.scripts.mps_example --model_name="mv3" --generate_etrecord -b diff --git a/docs/source/build-run-coreml.md b/docs/source/build-run-coreml.md index 52755773eed..6e0cc802df6 100644 --- a/docs/source/build-run-coreml.md +++ b/docs/source/build-run-coreml.md @@ -87,7 +87,7 @@ cd executorch Note that profiling is supported on [macOS](https://developer.apple.com/macos) >= 14.4. -1. [Optional] Generate an [ETRecord](./sdk-etrecord.rst) when exporting your model. +1. [Optional] Generate an [ETRecord](./etrecord.rst) when exporting your model. ```bash cd executorch @@ -108,7 +108,7 @@ cd executorch ./coreml_executor_runner --model_path mv3_coreml_all.pte --profile_model --etdump_path etdump.etdp ``` -4. Create an instance of the [Inspector API](./sdk-inspector.rst) by passing in the [ETDump](./sdk-etdump.md) you have sourced from the runtime along with the optionally generated [ETRecord](./sdk-etrecord.rst) from step 1 or execute the following command in your terminal to display the profiling data table. +4. Create an instance of the [Inspector API](./sdk-inspector.rst) by passing in the [ETDump](./sdk-etdump.md) you have sourced from the runtime along with the optionally generated [ETRecord](./etrecord.rst) from step 1 or execute the following command in your terminal to display the profiling data table. ```bash python examples/apple/coreml/scripts/inspector_cli.py --etdump_path etdump.etdp --etrecord_path mv3_coreml.bin ``` diff --git a/docs/source/compiler-delegate-and-partitioner.md b/docs/source/compiler-delegate-and-partitioner.md index c82af7d98fe..b5f7e0d3d8a 100644 --- a/docs/source/compiler-delegate-and-partitioner.md +++ b/docs/source/compiler-delegate-and-partitioner.md @@ -129,7 +129,7 @@ static auto success_with_compiler = register_backend(backend); ## Developer Tools Integration: Debuggability -Providing consistent debugging experience, be it for runtime failures or performance profiling, is important. ExecuTorch employs native Developer Tools for this purpose, which enables correlating program instructions to original PyTorch code, via debug handles. You can read more about it [here](./sdk-etrecord). +Providing consistent debugging experience, be it for runtime failures or performance profiling, is important. ExecuTorch employs native Developer Tools for this purpose, which enables correlating program instructions to original PyTorch code, via debug handles. You can read more about it [here](./etrecord). Delegated program or subgraphs are opaque to ExecuTorch runtime and appear as a special `call_delegate` instruction, which asks corresponding backend to handle the execution of the subgraph or program. Due to the opaque nature of backend delgates, native Developer Tools does not have visibility into delegated program. Thus the debugging, functional or performance, experiences of delegated execution suffers significantly as compared to it's non-delegated counterpart. diff --git a/docs/source/devtools-overview.md b/docs/source/devtools-overview.md index 771e8db6b95..e18b7f16c64 100644 --- a/docs/source/devtools-overview.md +++ b/docs/source/devtools-overview.md @@ -27,7 +27,7 @@ ETRecord (ExecuTorch Record) is an artifact generated during the export process To draw a rough equivalence to conventional software development ETRecord can be considered as the binary built with debug symbols that is used for debugging in GNU Project debugger (gdb). -More details are available in the [ETRecord documentation](sdk-etrecord.rst) on how to generate and store an ETRecord. +More details are available in the [ETRecord documentation](etrecord.rst) on how to generate and store an ETRecord. ### ETDump ETDump (ExecuTorch Dump) is the binary blob that is generated by the runtime after running a model. Similarly as above, to draw a rough equivalence to conventional software development, ETDump can be considered as the coredump of ExecuTorch, but in this case within ETDump we store all the performance and debug data that was generated by the runtime during model execution. diff --git a/docs/source/etrecord.rst b/docs/source/etrecord.rst new file mode 100644 index 00000000000..63546f43ca6 --- /dev/null +++ b/docs/source/etrecord.rst @@ -0,0 +1,40 @@ +Prerequisite | ETRecord - ExecuTorch Record +=========================================== + +Overview +-------- + +``ETRecord`` is intended to be the debug artifact that is generated by +users ahead of time (when they export their model to run on ExecuTorch). +To draw a rough equivalent to conventional software development, +``ETRecord`` can be considered as the binary built with debug symbols +that is used for debugging in GNU Debugger (gdb). It is expected that +the user will supply this to the ExecuTorch Developer Tools in order for +them to debug and visualize their model. + +``ETRecord`` contains numerous components such as: + +* Edge dialect graph with debug handles +* Delegate debug handle maps + +The ``ETRecord`` object itself is intended to be opaque to users and they should not access any components inside it directly. +It should be provided to the `Inspector API `__ to link back performance and debug data sourced from the runtime back to the Python source code. + +Generating an ``ETRecord`` +-------------------------- + +The user should use the following API to generate an ``ETRecord`` file. They +will be expected to provide the Edge Dialect program (returned by the call to ``to_edge()``), +the ExecuTorch program (returned by the call to ``to_executorch()``), and optional models that +they are interested in working with via our tooling. + +.. warning:: + Users should do a deepcopy of the output of ``to_edge()`` and pass in the deepcopy to the ``generate_etrecord`` API. This is needed because the subsequent call, ``to_executorch()``, does an in-place mutation and will lose debug data in the process. + +.. currentmodule:: executorch.devtools.etrecord._etrecord +.. autofunction:: generate_etrecord + +Using an ``ETRecord`` +--------------------- + +Pass the ``ETRecord`` as an optional argument into the `Inspector API `__ to access this data and do post-run analysis. diff --git a/docs/source/index.rst b/docs/source/index.rst index 20f0c944820..3573b4068ec 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -202,7 +202,7 @@ Topics in this section will help you get started with ExecuTorch. devtools-overview sdk-bundled-io - sdk-etrecord + etrecord sdk-etdump sdk-profiling sdk-debugging diff --git a/docs/source/llm/getting-started.md b/docs/source/llm/getting-started.md index 771bf489a94..1cfeab6e5e6 100644 --- a/docs/source/llm/getting-started.md +++ b/docs/source/llm/getting-started.md @@ -752,7 +752,7 @@ Through the ExecuTorch Developer Tools, users are able to profile model executio ##### ETRecord generation (Optional) -An ETRecord is an artifact generated at the time of export that contains model graphs and source-level metadata linking the ExecuTorch program to the original PyTorch model. You can view all profiling events without an ETRecord, though with an ETRecord, you will also be able to link each event to the types of operators being executed, module hierarchy, and stack traces of the original PyTorch source code. For more information, see [the ETRecord docs](../sdk-etrecord.md). +An ETRecord is an artifact generated at the time of export that contains model graphs and source-level metadata linking the ExecuTorch program to the original PyTorch model. You can view all profiling events without an ETRecord, though with an ETRecord, you will also be able to link each event to the types of operators being executed, module hierarchy, and stack traces of the original PyTorch source code. For more information, see [the ETRecord docs](../etrecord.md). In your export script, after calling `to_edge()` and `to_executorch()`, call `generate_etrecord()` with the `EdgeProgramManager` from `to_edge()` and the `ExecuTorchProgramManager` from `to_executorch()`. Make sure to copy the `EdgeProgramManager`, as the call to `to_backend()` mutates the graph in-place. diff --git a/docs/source/sdk-debugging.md b/docs/source/sdk-debugging.md index 4707b4a2f99..80358fe99a1 100644 --- a/docs/source/sdk-debugging.md +++ b/docs/source/sdk-debugging.md @@ -13,7 +13,7 @@ Currently, ExecuTorch supports the following debugging flows: ### Runtime For a real example reflecting the steps below, please refer to [example_runner.cpp](https://github.com/pytorch/executorch/blob/main/examples/devtools/example_runner/example_runner.cpp). -1. [Optional] Generate an [ETRecord](./sdk-etrecord.rst) while exporting your model. When provided, this enables users to link profiling information back to the eager model source code (with stack traces and module hierarchy). +1. [Optional] Generate an [ETRecord](./etrecord.rst) while exporting your model. When provided, this enables users to link profiling information back to the eager model source code (with stack traces and module hierarchy). 2. Integrate [ETDump generation](./sdk-etdump.md) into the runtime and set the debugging level by configuring the `ETDumpGen` object. Then, provide an additional buffer to which intermediate outputs and program outputs will be written. Currently we support two levels of debugging: - Program level outputs ```C++ diff --git a/docs/source/sdk-etrecord.rst b/docs/source/sdk-etrecord.rst index 63546f43ca6..ee8f9b2b2d2 100644 --- a/docs/source/sdk-etrecord.rst +++ b/docs/source/sdk-etrecord.rst @@ -1,40 +1,4 @@ Prerequisite | ETRecord - ExecuTorch Record =========================================== -Overview --------- - -``ETRecord`` is intended to be the debug artifact that is generated by -users ahead of time (when they export their model to run on ExecuTorch). -To draw a rough equivalent to conventional software development, -``ETRecord`` can be considered as the binary built with debug symbols -that is used for debugging in GNU Debugger (gdb). It is expected that -the user will supply this to the ExecuTorch Developer Tools in order for -them to debug and visualize their model. - -``ETRecord`` contains numerous components such as: - -* Edge dialect graph with debug handles -* Delegate debug handle maps - -The ``ETRecord`` object itself is intended to be opaque to users and they should not access any components inside it directly. -It should be provided to the `Inspector API `__ to link back performance and debug data sourced from the runtime back to the Python source code. - -Generating an ``ETRecord`` --------------------------- - -The user should use the following API to generate an ``ETRecord`` file. They -will be expected to provide the Edge Dialect program (returned by the call to ``to_edge()``), -the ExecuTorch program (returned by the call to ``to_executorch()``), and optional models that -they are interested in working with via our tooling. - -.. warning:: - Users should do a deepcopy of the output of ``to_edge()`` and pass in the deepcopy to the ``generate_etrecord`` API. This is needed because the subsequent call, ``to_executorch()``, does an in-place mutation and will lose debug data in the process. - -.. currentmodule:: executorch.devtools.etrecord._etrecord -.. autofunction:: generate_etrecord - -Using an ``ETRecord`` ---------------------- - -Pass the ``ETRecord`` as an optional argument into the `Inspector API `__ to access this data and do post-run analysis. +Please update your link to . This URL will be deleted after v0.4.0. diff --git a/docs/source/sdk-inspector.rst b/docs/source/sdk-inspector.rst index 4f55271b3fe..4d46915a8af 100644 --- a/docs/source/sdk-inspector.rst +++ b/docs/source/sdk-inspector.rst @@ -5,7 +5,7 @@ Overview -------- The Inspector APIs provide a convenient interface for analyzing the -contents of `ETRecord `__ and +contents of `ETRecord `__ and `ETDump `__, helping developers get insights about model architecture and performance statistics. It’s built on top of the `EventBlock Class <#eventblock-class>`__ data structure, which organizes a group of `Event <#event-class>`__\ s for easy access to details of profiling events. diff --git a/docs/source/sdk-profiling.md b/docs/source/sdk-profiling.md index e17fb1ae48e..945260721e7 100644 --- a/docs/source/sdk-profiling.md +++ b/docs/source/sdk-profiling.md @@ -13,7 +13,7 @@ We provide access to all the profiling data via the Python [Inspector API](./sdk ## Steps to Profile a Model in ExecuTorch -1. [Optional] Generate an [ETRecord](./sdk-etrecord.rst) while you're exporting your model. If provided this will enable users to link back profiling details to eager model source code (with stack traces and module hierarchy). +1. [Optional] Generate an [ETRecord](./etrecord.rst) while you're exporting your model. If provided this will enable users to link back profiling details to eager model source code (with stack traces and module hierarchy). 2. Build the runtime with the pre-processor flags that enable profiling. Detailed in the [ETDump documentation](./sdk-etdump.md). 3. Run your Program on the ExecuTorch runtime and generate an [ETDump](./sdk-etdump.md). 4. Create an instance of the [Inspector API](./sdk-inspector.rst) by passing in the ETDump you have sourced from the runtime along with the optionally generated ETRecord from step 1. diff --git a/docs/source/tutorials_source/devtools-integration-tutorial.py b/docs/source/tutorials_source/devtools-integration-tutorial.py index b5e335b43d1..642805fc33a 100644 --- a/docs/source/tutorials_source/devtools-integration-tutorial.py +++ b/docs/source/tutorials_source/devtools-integration-tutorial.py @@ -20,7 +20,7 @@ # This tutorial will show a full end-to-end flow of how to utilize the Developer Tools to profile a model. # Specifically, it will: # -# 1. Generate the artifacts consumed by the Developer Tools (`ETRecord <../sdk-etrecord.html>`__, `ETDump <../sdk-etdump.html>`__). +# 1. Generate the artifacts consumed by the Developer Tools (`ETRecord <../etrecord.html>`__, `ETDump <../sdk-etdump.html>`__). # 2. Create an Inspector class consuming these artifacts. # 3. Utilize the Inspector class to analyze the model profiling result. @@ -296,6 +296,6 @@ def forward(self, x): # ^^^^^^^^^^^^^^^ # # - `ExecuTorch Developer Tools Overview <../devtools-overview.html>`__ -# - `ETRecord <../sdk-etrecord.html>`__ +# - `ETRecord <../etrecord.html>`__ # - `ETDump <../sdk-etdump.html>`__ # - `Inspector <../sdk-inspector.html>`__