Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion backends/apple/mps/setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@ python3 -m examples.apple.mps.scripts.mps_example --model_name="mv3" --no-use_fp
```

### Profiling:
1. [Optional] Generate an [ETRecord](./sdk-etrecord.rst) while you're exporting your model.
1. [Optional] Generate an [ETRecord](./etrecord.rst) while you're exporting your model.
```bash
cd executorch
python3 -m examples.apple.mps.scripts.mps_example --model_name="mv3" --generate_etrecord -b
Expand Down
4 changes: 2 additions & 2 deletions docs/source/build-run-coreml.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ cd executorch

Note that profiling is supported on [macOS](https://developer.apple.com/macos) >= 14.4.

1. [Optional] Generate an [ETRecord](./sdk-etrecord.rst) when exporting your model.
1. [Optional] Generate an [ETRecord](./etrecord.rst) when exporting your model.
```bash
cd executorch

Expand All @@ -108,7 +108,7 @@ cd executorch
./coreml_executor_runner --model_path mv3_coreml_all.pte --profile_model --etdump_path etdump.etdp
```

4. Create an instance of the [Inspector API](./sdk-inspector.rst) by passing in the [ETDump](./sdk-etdump.md) you have sourced from the runtime along with the optionally generated [ETRecord](./sdk-etrecord.rst) from step 1 or execute the following command in your terminal to display the profiling data table.
4. Create an instance of the [Inspector API](./sdk-inspector.rst) by passing in the [ETDump](./sdk-etdump.md) you have sourced from the runtime along with the optionally generated [ETRecord](./etrecord.rst) from step 1 or execute the following command in your terminal to display the profiling data table.
```bash
python examples/apple/coreml/scripts/inspector_cli.py --etdump_path etdump.etdp --etrecord_path mv3_coreml.bin
```
Expand Down
2 changes: 1 addition & 1 deletion docs/source/compiler-delegate-and-partitioner.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,7 @@ static auto success_with_compiler = register_backend(backend);

## Developer Tools Integration: Debuggability

Providing consistent debugging experience, be it for runtime failures or performance profiling, is important. ExecuTorch employs native Developer Tools for this purpose, which enables correlating program instructions to original PyTorch code, via debug handles. You can read more about it [here](./sdk-etrecord).
Providing consistent debugging experience, be it for runtime failures or performance profiling, is important. ExecuTorch employs native Developer Tools for this purpose, which enables correlating program instructions to original PyTorch code, via debug handles. You can read more about it [here](./etrecord).

Delegated program or subgraphs are opaque to ExecuTorch runtime and appear as a special `call_delegate` instruction, which asks corresponding backend to handle the execution of the subgraph or program. Due to the opaque nature of backend delgates, native Developer Tools does not have visibility into delegated program. Thus the debugging, functional or performance, experiences of delegated execution suffers significantly as compared to it's non-delegated counterpart.

Expand Down
2 changes: 1 addition & 1 deletion docs/source/devtools-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ ETRecord (ExecuTorch Record) is an artifact generated during the export process

To draw a rough equivalence to conventional software development ETRecord can be considered as the binary built with debug symbols that is used for debugging in GNU Project debugger (gdb).

More details are available in the [ETRecord documentation](sdk-etrecord.rst) on how to generate and store an ETRecord.
More details are available in the [ETRecord documentation](etrecord.rst) on how to generate and store an ETRecord.

### ETDump
ETDump (ExecuTorch Dump) is the binary blob that is generated by the runtime after running a model. Similarly as above, to draw a rough equivalence to conventional software development, ETDump can be considered as the coredump of ExecuTorch, but in this case within ETDump we store all the performance and debug data that was generated by the runtime during model execution.
Expand Down
40 changes: 40 additions & 0 deletions docs/source/etrecord.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
Prerequisite | ETRecord - ExecuTorch Record
===========================================

Overview
--------

``ETRecord`` is intended to be the debug artifact that is generated by
users ahead of time (when they export their model to run on ExecuTorch).
To draw a rough equivalent to conventional software development,
``ETRecord`` can be considered as the binary built with debug symbols
that is used for debugging in GNU Debugger (gdb). It is expected that
the user will supply this to the ExecuTorch Developer Tools in order for
them to debug and visualize their model.

``ETRecord`` contains numerous components such as:

* Edge dialect graph with debug handles
* Delegate debug handle maps

The ``ETRecord`` object itself is intended to be opaque to users and they should not access any components inside it directly.
It should be provided to the `Inspector API <sdk-inspector.html>`__ to link back performance and debug data sourced from the runtime back to the Python source code.

Generating an ``ETRecord``
--------------------------

The user should use the following API to generate an ``ETRecord`` file. They
will be expected to provide the Edge Dialect program (returned by the call to ``to_edge()``),
the ExecuTorch program (returned by the call to ``to_executorch()``), and optional models that
they are interested in working with via our tooling.

.. warning::
Users should do a deepcopy of the output of ``to_edge()`` and pass in the deepcopy to the ``generate_etrecord`` API. This is needed because the subsequent call, ``to_executorch()``, does an in-place mutation and will lose debug data in the process.

.. currentmodule:: executorch.devtools.etrecord._etrecord
.. autofunction:: generate_etrecord

Using an ``ETRecord``
---------------------

Pass the ``ETRecord`` as an optional argument into the `Inspector API <sdk-inspector.html>`__ to access this data and do post-run analysis.
2 changes: 1 addition & 1 deletion docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -202,7 +202,7 @@ Topics in this section will help you get started with ExecuTorch.

devtools-overview
sdk-bundled-io
sdk-etrecord
etrecord
sdk-etdump
sdk-profiling
sdk-debugging
Expand Down
2 changes: 1 addition & 1 deletion docs/source/llm/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -752,7 +752,7 @@ Through the ExecuTorch Developer Tools, users are able to profile model executio

##### ETRecord generation (Optional)

An ETRecord is an artifact generated at the time of export that contains model graphs and source-level metadata linking the ExecuTorch program to the original PyTorch model. You can view all profiling events without an ETRecord, though with an ETRecord, you will also be able to link each event to the types of operators being executed, module hierarchy, and stack traces of the original PyTorch source code. For more information, see [the ETRecord docs](../sdk-etrecord.md).
An ETRecord is an artifact generated at the time of export that contains model graphs and source-level metadata linking the ExecuTorch program to the original PyTorch model. You can view all profiling events without an ETRecord, though with an ETRecord, you will also be able to link each event to the types of operators being executed, module hierarchy, and stack traces of the original PyTorch source code. For more information, see [the ETRecord docs](../etrecord.md).


In your export script, after calling `to_edge()` and `to_executorch()`, call `generate_etrecord()` with the `EdgeProgramManager` from `to_edge()` and the `ExecuTorchProgramManager` from `to_executorch()`. Make sure to copy the `EdgeProgramManager`, as the call to `to_backend()` mutates the graph in-place.
Expand Down
2 changes: 1 addition & 1 deletion docs/source/sdk-debugging.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Currently, ExecuTorch supports the following debugging flows:
### Runtime
For a real example reflecting the steps below, please refer to [example_runner.cpp](https://github.com/pytorch/executorch/blob/main/examples/devtools/example_runner/example_runner.cpp).

1. [Optional] Generate an [ETRecord](./sdk-etrecord.rst) while exporting your model. When provided, this enables users to link profiling information back to the eager model source code (with stack traces and module hierarchy).
1. [Optional] Generate an [ETRecord](./etrecord.rst) while exporting your model. When provided, this enables users to link profiling information back to the eager model source code (with stack traces and module hierarchy).
2. Integrate [ETDump generation](./sdk-etdump.md) into the runtime and set the debugging level by configuring the `ETDumpGen` object. Then, provide an additional buffer to which intermediate outputs and program outputs will be written. Currently we support two levels of debugging:
- Program level outputs
```C++
Expand Down
38 changes: 1 addition & 37 deletions docs/source/sdk-etrecord.rst
Original file line number Diff line number Diff line change
@@ -1,40 +1,4 @@
Prerequisite | ETRecord - ExecuTorch Record
===========================================

Overview
--------

``ETRecord`` is intended to be the debug artifact that is generated by
users ahead of time (when they export their model to run on ExecuTorch).
To draw a rough equivalent to conventional software development,
``ETRecord`` can be considered as the binary built with debug symbols
that is used for debugging in GNU Debugger (gdb). It is expected that
the user will supply this to the ExecuTorch Developer Tools in order for
them to debug and visualize their model.

``ETRecord`` contains numerous components such as:

* Edge dialect graph with debug handles
* Delegate debug handle maps

The ``ETRecord`` object itself is intended to be opaque to users and they should not access any components inside it directly.
It should be provided to the `Inspector API <sdk-inspector.html>`__ to link back performance and debug data sourced from the runtime back to the Python source code.

Generating an ``ETRecord``
--------------------------

The user should use the following API to generate an ``ETRecord`` file. They
will be expected to provide the Edge Dialect program (returned by the call to ``to_edge()``),
the ExecuTorch program (returned by the call to ``to_executorch()``), and optional models that
they are interested in working with via our tooling.

.. warning::
Users should do a deepcopy of the output of ``to_edge()`` and pass in the deepcopy to the ``generate_etrecord`` API. This is needed because the subsequent call, ``to_executorch()``, does an in-place mutation and will lose debug data in the process.

.. currentmodule:: executorch.devtools.etrecord._etrecord
.. autofunction:: generate_etrecord

Using an ``ETRecord``
---------------------

Pass the ``ETRecord`` as an optional argument into the `Inspector API <sdk-inspector.html>`__ to access this data and do post-run analysis.
Please update your link to <https://pytorch.org/executorch/main/etrecord.html>. This URL will be deleted after v0.4.0.
2 changes: 1 addition & 1 deletion docs/source/sdk-inspector.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ Overview
--------

The Inspector APIs provide a convenient interface for analyzing the
contents of `ETRecord <sdk-etrecord.html>`__ and
contents of `ETRecord <etrecord.html>`__ and
`ETDump <sdk-etdump.html>`__, helping developers get insights about model
architecture and performance statistics. It’s built on top of the `EventBlock Class <#eventblock-class>`__ data structure,
which organizes a group of `Event <#event-class>`__\ s for easy access to details of profiling events.
Expand Down
2 changes: 1 addition & 1 deletion docs/source/sdk-profiling.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ We provide access to all the profiling data via the Python [Inspector API](./sdk

## Steps to Profile a Model in ExecuTorch

1. [Optional] Generate an [ETRecord](./sdk-etrecord.rst) while you're exporting your model. If provided this will enable users to link back profiling details to eager model source code (with stack traces and module hierarchy).
1. [Optional] Generate an [ETRecord](./etrecord.rst) while you're exporting your model. If provided this will enable users to link back profiling details to eager model source code (with stack traces and module hierarchy).
2. Build the runtime with the pre-processor flags that enable profiling. Detailed in the [ETDump documentation](./sdk-etdump.md).
3. Run your Program on the ExecuTorch runtime and generate an [ETDump](./sdk-etdump.md).
4. Create an instance of the [Inspector API](./sdk-inspector.rst) by passing in the ETDump you have sourced from the runtime along with the optionally generated ETRecord from step 1.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
# This tutorial will show a full end-to-end flow of how to utilize the Developer Tools to profile a model.
# Specifically, it will:
#
# 1. Generate the artifacts consumed by the Developer Tools (`ETRecord <../sdk-etrecord.html>`__, `ETDump <../sdk-etdump.html>`__).
# 1. Generate the artifacts consumed by the Developer Tools (`ETRecord <../etrecord.html>`__, `ETDump <../sdk-etdump.html>`__).
# 2. Create an Inspector class consuming these artifacts.
# 3. Utilize the Inspector class to analyze the model profiling result.

Expand Down Expand Up @@ -296,6 +296,6 @@ def forward(self, x):
# ^^^^^^^^^^^^^^^
#
# - `ExecuTorch Developer Tools Overview <../devtools-overview.html>`__
# - `ETRecord <../sdk-etrecord.html>`__
# - `ETRecord <../etrecord.html>`__
# - `ETDump <../sdk-etdump.html>`__
# - `Inspector <../sdk-inspector.html>`__
Loading