Arm backend: Remove TOSA v0.80 from arm backend #13040

SaoirseARM · 2025-07-31T12:34:09Z

Summary

This PR removes TOSA v0.80 from the arm backend. This includes updates to:

operator node visitors
operator support
passes
test infrastructure
tosa specification and utils
removal of installation of v0.80 and patch scripts

cc @digantdesai @freddan80 @per @zingo @oscarandersson8218

Change-Id: Ie3442b53184ecf0fbf817b581b780d2456670c4f

Change-Id: I21503bcab233d813759132b4c5a33dc6f7ed4f13

Change-Id: I9ff98f6560c686527cba18c28ea98c00683a3c03

- Update test names to use INT/FP - Update the test name checker script for INT/FP Change-Id: I93acbd0e5f0c6bca6095fb39f267ed7a01bf0796

Change-Id: I183a1475ace4c4fe73ab325d615974464d3520d3

- Update README - Remove tosa 0.80 schema - Remove tosa 0.80 patches Change-Id: I81f88d5af6e14681ac09c2c6d51f4adbfd081aeb

pytorch-bot · 2025-07-31T12:34:12Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/13040

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures

As of commit 8c2af87 with merge base bedce91 ():

NEW FAILURES - The following jobs have failed:

pull / test-llava-runner-linux / linux-job (gh)
RuntimeError: Command docker exec -t 336a0bb9ed6332ad5e4722b62245d4a74690e8429f3155d8c13b9e4f1b7af36e /exec failed with exit code 139
trunk / test-arm-cortex-m-size-test (bare_metal) / linux-job (gh)
RuntimeError: Command docker exec -t 1bfde1d80f2b154f452aa82eb112c7ce43fa57bbd88c1ab39b3052053c27c36f /exec failed with exit code 1
trunk / test-llama-runner-mac (fp32, coreml) / macos-job (gh)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

SaoirseARM · 2025-07-31T12:34:32Z

@pytorchbot label ciflow/trunk

SaoirseARM · 2025-07-31T12:35:32Z

@pytorchbot label 'partner: arm'

SaoirseARM · 2025-07-31T12:50:17Z

@pytorchbot label "release notes: none"

zingo · 2025-07-31T14:35:05Z

cortex-m fails independent of this and is related to C/C++ code size
llava/lama test is unrelated

kirklandsign · 2025-08-04T05:19:38Z

backends/arm/test/runner_utils.py

-        )
-    elif isinstance(tosa_version, Tosa_1_00):
+    if isinstance(tosa_version, Tosa_1_00):
        import tosa_reference_model as reference_model


Got error

> import tosa_reference_model as reference_model E ModuleNotFoundError: No module named 'tosa_reference_model'

yeah we shouldn't assume reference model is available, we should use it only if its "available"

kirklandsign · 2025-08-04T05:20:05Z

Hi @zingo @SaoirseARM seems that this test is failing. Could you please take a look?

zingo · 2025-08-04T07:30:56Z

Hi thanks for reporting, we will look into it, do you have a link to where you see the problem, we don't see it internally and when I look an the "hud" https://hud.pytorch.org/hud/pytorch/executorch/main it seem to not be a problems.

Are you running the tests locally, you might need to rerun setup.sh to re-setup all stuff after the removal maybe?
e.g.
examples/arm/setup.sh --i-agree-to-the-contained-eula

kirklandsign · 2025-08-04T16:46:16Z

executorch/backends/arm/test  ARM_TEST_SEED=323055682  F

=================================== FAILURES ===================================
_________________ test_not_fuse_tensor_with_different_type_FP __________________

    def test_not_fuse_tensor_with_different_type_FP():
        module = NotFuseTensorWithDifferentType()
        data = (
            torch.rand(
                1,
            ),
            torch.randint(
                0,
                10,
                (1,),
                dtype=torch.int,
            ),
        )
        pipeline = TosaPipelineFP[input_t](
            module,
            data,
            aten_op=[],
        )
>       pipeline.run()

executorch/backends/arm/test/__fuse_equal_placeholders_ops_pass__/fuse_equal_placeholders_ops_pass#link-tree/executorch/backends/arm/test/passes/test_fuse_equal_placeholders_ops_pass.py:139: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
executorch/backends/arm/test/__fuse_equal_placeholders_ops_pass__/fuse_equal_placeholders_ops_pass#link-tree/executorch/backends/arm/test/tester/test_pipeline.py:271: in run
    raise e
executorch/backends/arm/test/__fuse_equal_placeholders_ops_pass__/fuse_equal_placeholders_ops_pass#link-tree/executorch/backends/arm/test/tester/test_pipeline.py:268: in run
    stage()
executorch/backends/arm/test/__fuse_equal_placeholders_ops_pass__/fuse_equal_placeholders_ops_pass#link-tree/executorch/backends/arm/test/tester/test_pipeline.py:85: in __call__
    self.func(*self.args, **self.kwargs)
executorch/backends/arm/test/__fuse_equal_placeholders_ops_pass__/fuse_equal_placeholders_ops_pass#link-tree/executorch/backends/arm/test/tester/arm_tester.py:521: in run_method_and_compare_outputs
    test_stage.run_artifact(reference_input)
executorch/backends/arm/test/__fuse_equal_placeholders_ops_pass__/fuse_equal_placeholders_ops_pass#link-tree/executorch/backends/arm/test/tester/arm_tester.py:226: in run_artifact
    return super().run_artifact(inputs)
executorch/backends/arm/test/__fuse_equal_placeholders_ops_pass__/fuse_equal_placeholders_ops_pass#link-tree/executorch/backends/test/harness/stages/stage.py:64: in run_artifact
    return self.artifact.exported_program().module()(*inputs)
executorch/backends/arm/test/__fuse_equal_placeholders_ops_pass__/fuse_equal_placeholders_ops_pass#link-tree/torch/fx/graph_module.py:850: in call_wrapped
    return self._wrapped_call(self, *args, **kwargs)
executorch/backends/arm/test/__fuse_equal_placeholders_ops_pass__/fuse_equal_placeholders_ops_pass#link-tree/torch/fx/graph_module.py:426: in __call__
    raise e
executorch/backends/arm/test/__fuse_equal_placeholders_ops_pass__/fuse_equal_placeholders_ops_pass#link-tree/torch/fx/graph_module.py:413: in __call__
    return super(self.cls, obj).__call__(*args, **kwargs)  # type: ignore[misc]
executorch/backends/arm/test/__fuse_equal_placeholders_ops_pass__/fuse_equal_placeholders_ops_pass#link-tree/torch/nn/modules/module.py:1773: in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
executorch/backends/arm/test/__fuse_equal_placeholders_ops_pass__/fuse_equal_placeholders_ops_pass#link-tree/torch/nn/modules/module.py:1879: in _call_impl
    return inner()
executorch/backends/arm/test/__fuse_equal_placeholders_ops_pass__/fuse_equal_placeholders_ops_pass#link-tree/torch/nn/modules/module.py:1827: in inner
    result = forward_call(*args, **kwargs)
<eval_with_key>.350:7: in forward
    executorch_call_delegate = torch.ops.higher_order.executorch_call_delegate(lowered_module_0, x, y);  lowered_module_0 = x = y = None
executorch/backends/arm/test/__fuse_equal_placeholders_ops_pass__/fuse_equal_placeholders_ops_pass#link-tree/torch/_higher_order_ops/executorch_call_delegate.py:33: in __call__
    return super().__call__(lowered_module, *args)
executorch/backends/arm/test/__fuse_equal_placeholders_ops_pass__/fuse_equal_placeholders_ops_pass#link-tree/torch/_ops.py:535: in __call__
    return wrapper()
executorch/backends/arm/test/__fuse_equal_placeholders_ops_pass__/fuse_equal_placeholders_ops_pass#link-tree/torch/_ops.py:526: in wrapper
    return torch.overrides.handle_torch_function(
executorch/backends/arm/test/__fuse_equal_placeholders_ops_pass__/fuse_equal_placeholders_ops_pass#link-tree/torch/overrides.py:1727: in handle_torch_function
    result = mode.__torch_function__(public_api, types, args, kwargs)
executorch/backends/arm/test/__fuse_equal_placeholders_ops_pass__/fuse_equal_placeholders_ops_pass#link-tree/executorch/backends/arm/test/runner_utils.py:208: in __torch_function__
    return self._tosa_dispatch(lowered_backend_module, args[1:])
executorch/backends/arm/test/__fuse_equal_placeholders_ops_pass__/fuse_equal_placeholders_ops_pass#link-tree/executorch/backends/arm/test/runner_utils.py:193: in _tosa_dispatch
    return run_tosa_graph(tosa_buffer, tosa_spec, inputs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

graph = b'\x0c\x00\x00\x00TOSA\x00\x00\x00\x00\xec\xff\xff\xff\x08\x00\x00\x004\x04\x00\x00\x01\x00\x00\x00\x0c\x00\x00\x00\x0...00\x00main\x00\x00
\x00\x10\x00\x0c\x00\x08\x00\x04\x00
\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00'
tosa_version = TOSA-1.0+FP
inputs = (tensor([0.6266]), tensor([6], dtype=torch.int32))

    def run_tosa_graph(
        graph: Any,
        tosa_version: TosaSpecification,
        inputs: list[torch.Tensor],
    ) -> list[torch.Tensor]:
        """Runs the TOSA reference model with inputs and returns the result."""
        inputs_np = [input.numpy() for input in inputs]
        transpose_data_format(inputs_np, to="NHWC")
    
        if isinstance(tosa_version, Tosa_1_00):
>           import tosa_reference_model as reference_model
E           ModuleNotFoundError: No module named 'tosa_reference_model'

kirklandsign · 2025-08-05T07:07:08Z

Hi @zingo any suggestions on this issue?

zingo · 2025-08-05T07:24:48Z

Hi Im still trying to reprodude and for me this works, so I have to dig some deeper

╰─❯ pytest  --verbose --color=yes backends/arm/test/passes/test_fuse_equal_placeholders_ops_pass.py 
============================================================================================= test session starts =============================================================================================
platform linux -- Python 3.12.3, pytest-8.3.5, pluggy-1.5.0 -- /home/zinand01/Projects/ml/executorch/executorch/env/bin/python3
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase(PosixPath('/home/zinand01/Projects/ml/executorch/executorch/.hypothesis/examples'))
rootdir: /home/zinand01/Projects/ml/executorch/executorch/backends/arm/test
configfile: pytest.ini
plugins: hydra-core-1.3.2, xdist-3.6.1, hypothesis-6.130.4, rerunfailures-15.0
collected 3 items                                                                                                                                                                                             

backends/arm/test/passes/test_fuse_equal_placeholders_ops_pass.py::test_fuse_equal_placeholders_constants_tosa_FP PASSED                                                                                [ 33%]
backends/arm/test/passes/test_fuse_equal_placeholders_ops_pass.py::test_fuse_equal_placeholders_state_dict_tosa_FP PASSED                                                                               [ 66%]
backends/arm/test/passes/test_fuse_equal_placeholders_ops_pass.py::test_not_fuse_tensor_with_different_type_FP PASSED  

============================================================================================== warnings summary ===============================================================================================
env/lib/python3.12/site-packages/executorch/exir/dialects/edge/_ops.py:9
  /home/zinand01/Projects/ml/executorch/executorch/env/lib/python3.12/site-packages/executorch/exir/dialects/edge/_ops.py:9: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    import pkg_resources

env/lib/python3.12/site-packages/executorch/extension/pybindings/portable_lib.py:20
  /home/zinand01/Projects/ml/executorch/executorch/env/lib/python3.12/site-packages/executorch/extension/pybindings/portable_lib.py:20: ExperimentalWarning: This API is experimental and subject to change without notice.
    _warnings.warn(

passes/test_fuse_equal_placeholders_ops_pass.py::test_not_fuse_tensor_with_different_type_FP
passes/test_fuse_equal_placeholders_ops_pass.py::test_not_fuse_tensor_with_different_type_FP
  /home/zinand01/Projects/ml/executorch/executorch/env/lib/python3.12/site-packages/executorch/exir/_serialize/_flatbuffer.py:105: DeprecationWarning: read_binary is deprecated. Use files() instead. Refer to https://importlib-resources.readthedocs.io/en/latest/using.html#migrating-from-legacy for migration advice.
    self._files[name] = importlib.resources.read_binary(__package__, name)

passes/test_fuse_equal_placeholders_ops_pass.py::test_not_fuse_tensor_with_different_type_FP
  /home/zinand01/Projects/ml/executorch/executorch/env/lib/python3.12/site-packages/executorch/exir/_serialize/_flatbuffer.py:190: DeprecationWarning: is_resource is deprecated. Use files() instead. Refer to https://importlib-resources.readthedocs.io/en/latest/using.html#migrating-from-legacy for migration advice.
    if importlib.resources.is_resource(__package__, _FLATC_RESOURCE_NAME):

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
======================================================================================== 3 passed, 5 warnings in 7.09s ========================================================================================

Are you seeing this in a CI (or something that has been setup from scratch)? Or on a dev machine (that has migrated)
I wonder if there is a rest of some 0.80.1 somewhere in some pip-out folder or something else. rerunning our setup.sh should have installed the reference model.
e.g. maybe there are some scratch or build folder that got some old stuff mixed in and needs to be wiped.

zingo · 2025-08-05T08:21:37Z

@kirklandsign do you get this on all tests? e.g. not only test_fuse_equal_placeholders_ops_pass.py

SaoirseARM · 2025-08-05T10:38:41Z

Hi @kirklandsign ,

It looks like the reference model has not installed correctly into the environment which seems to be why the serializer works and the reference model does not.

Do you use examples/arm/setup.sh --i-agree-to-the-contained-eula to set up the TOSA tools? If that is the case, I can provide a solution to ensure the reference model does indeed install.

Let me know if that would help.

Thanks,
Saoirse

kirklandsign · 2025-08-05T18:03:43Z

Are you seeing this in a CI (or something that has been setup from scratch)? Or on a dev machine (that has migrated)

In a CI. I'm not sure how the setup is done.

@kirklandsign do you get this on all tests? e.g. not only test_fuse_equal_placeholders_ops_pass.py

I get this on all tests

Do you use examples/arm/setup.sh --i-agree-to-the-contained-eula to set up the TOSA tools? If that is the case, I can provide a solution to ensure the reference model does indeed install.

Sorry I'm not sure how it is set up. Maybe @digantdesai knows

digantdesai · 2025-08-05T18:31:03Z

@kirklandsign is it breaking CI internally?

digantdesai · 2025-08-06T12:20:35Z

Let me fwd fix it

zingo · 2025-08-06T14:08:44Z

Thanks, Im still trying to reproduce and also try to great some github jobs with this not installed.

Some reflection:
Does it really make sense to run the unit tests in the case when it's not installed ? Would the run.sh flow be good enough? Thinking that a reference model is more of the unit test requirement but for the end 2 end flow (setup.sh) it should/could still work without it.

Would be OK to share how you install all TOSA stuff and Ill try to make a github test as close as possible. Then maybe we can see and fix this kind of things before merging :)

We are using this in the project (and not github testing)
https://github.com/pytorch/executorch/blob/main/backends/arm/scripts/install_reference_model.sh

After running it I see

╰─❯ pip list | grep tosa
tosa-tools                     1.0.0rc0.post1.dev314+g1e6e452

digantdesai · 2025-08-06T18:46:41Z

Does it really make sense to run the unit tests in the case when it's not installed ?

Yes! Because we can test the lowering until the PTE generation. Adding the tosa_reference_model in our internal setup has been on my list for a while so hopefully soon.

Would be OK to share how you install all TOSA stuff and Ill try to make a github test as close as possible

We use buck internally, so can't use CMake. And can't use install_reference_model.sh either.

digantdesai · 2025-08-06T18:50:00Z

FYI #13165 #13165

### Summary This PR removes TOSA v0.80 from the arm backend. This includes updates to: - operator node visitors - operator support - passes - test infrastructure - tosa specification and utils - removal of installation of v0.80 and patch scripts

SaoirseARM added 6 commits July 31, 2025 13:26

Arm backend: Remove TOSA 0.80 from operator node visitors

104c31b

Change-Id: Ie3442b53184ecf0fbf817b581b780d2456670c4f

Arm backend: Remove TOSA 0.80 from operator support

4c50d97

Change-Id: I21503bcab233d813759132b4c5a33dc6f7ed4f13

Arm backend: Remove TOSA 0.80 from passes

26a3584

Change-Id: I9ff98f6560c686527cba18c28ea98c00683a3c03

Arm backend: Remove TOSA 0.80 from test infrastructure

fec01d2

- Update test names to use INT/FP - Update the test name checker script for INT/FP Change-Id: I93acbd0e5f0c6bca6095fb39f267ed7a01bf0796

Arm backend: Remove TOSA 0.80 from tosa specification and utils

dd2d9f2

Change-Id: I183a1475ace4c4fe73ab325d615974464d3520d3

Arm backend: Remove install of TOSA 0.80

8c2af87

- Update README - Remove tosa 0.80 schema - Remove tosa 0.80 patches Change-Id: I81f88d5af6e14681ac09c2c6d51f4adbfd081aeb

SaoirseARM requested a review from zingo July 31, 2025 12:34

SaoirseARM requested a review from digantdesai as a code owner July 31, 2025 12:34

meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Jul 31, 2025

pytorch-bot bot added the ciflow/trunk label Jul 31, 2025

pytorch-bot bot added the partner: arm For backend delegation, kernels, demo, etc. from the 3rd-party partner, Arm label Jul 31, 2025

pytorch-bot bot added the release notes: none Do not include this in the release notes label Jul 31, 2025

zingo approved these changes Jul 31, 2025

View reviewed changes

zingo merged commit 2aa8084 into pytorch:main Jul 31, 2025
224 of 229 checks passed

kirklandsign reviewed Aug 4, 2025

View reviewed changes

Arm backend: Remove TOSA v0.80 from arm backend #13040

Arm backend: Remove TOSA v0.80 from arm backend #13040

Uh oh!

Conversation

SaoirseARM commented Jul 31, 2025 • edited by pytorch-bot bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

pytorch-bot bot commented Jul 31, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/13040

❌ 3 New Failures

Uh oh!

SaoirseARM commented Jul 31, 2025

Uh oh!

SaoirseARM commented Jul 31, 2025

Uh oh!

SaoirseARM commented Jul 31, 2025

Uh oh!

zingo commented Jul 31, 2025

Uh oh!

Uh oh!

kirklandsign Aug 4, 2025

Choose a reason for hiding this comment

Uh oh!

digantdesai Aug 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kirklandsign commented Aug 4, 2025

Uh oh!

zingo commented Aug 4, 2025

Uh oh!

kirklandsign commented Aug 4, 2025

Uh oh!

kirklandsign commented Aug 5, 2025

Uh oh!

zingo commented Aug 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zingo commented Aug 5, 2025

Uh oh!

SaoirseARM commented Aug 5, 2025

Uh oh!

kirklandsign commented Aug 5, 2025

Uh oh!

digantdesai commented Aug 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

digantdesai commented Aug 6, 2025

Uh oh!

zingo commented Aug 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

digantdesai commented Aug 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

digantdesai commented Aug 6, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

SaoirseARM commented Jul 31, 2025 •

edited by pytorch-bot bot

Loading

pytorch-bot bot commented Jul 31, 2025 •

edited

Loading

digantdesai Aug 5, 2025 •

edited

Loading

zingo commented Aug 5, 2025 •

edited

Loading

digantdesai commented Aug 5, 2025 •

edited

Loading

zingo commented Aug 6, 2025 •

edited

Loading

digantdesai commented Aug 6, 2025 •

edited

Loading