Executorch compilation error

[podraco]

greetings, i've been following the tutorials and other documentation to try and build executorch, but i faced multiple issues, right now, the issue i'm facing is that _cslt_sparse_mm_search is throwing me an error on compilation, this might be due to some error on typing. which can mean that there is an issue with versioning.

This is the command i use when i try to build the executorch for linux X86_64

cmake --build cmake-out -j9

This is the command i use after configuring the package to build for android (aarch64)

cmake --build . -j8

These are the documents i followed.

https://pytorch.org/executorch/stable/build-run-qualcomm-ai-engine-direct-backend.html
https://pytorch.org/executorch/stable/getting-started-setup.html#building-a-runtime
https://pytorch.org/executorch/stable/runtime-build-and-cross-compilation.html

this is the output i'm getting.

[ 31%] Generating code for kernel registration
[ 97%] Built target portable_kernels
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "$PYTHON3_11_LINUXx86_64_env/lib/python3.11/site-packages/torchgen/gen_executorch.py", line 943, in <module>
    main()
  File "$PYTHON3_11_LINUXx86_64_env/lib/python3.11/site-packages/torchgen/gen_executorch.py", line 882, in main
    parsed_yaml, custom_ops_parsed_yaml = parse_yaml_files(
                                          ^^^^^^^^^^^^^^^^^
  File "$PYTHON3_11_LINUXx86_64_env/lib/python3.11/site-packages/torchgen/gen_executorch.py", line 747, in parse_yaml_files
    translate_native_yaml(
  File "$PYTHON3_11_LINUXx86_64_env/lib/python3.11/site-packages/torchgen/gen_executorch.py", line 599, in translate_native_yaml
    native_functions, persisted_fields = parse_et_yaml(
                                         ^^^^^^^^^^^^^^
  File "$PYTHON3_11_LINUXx86_64_env/lib/python3.11/site-packages/torchgen/executorch/parse.py", line 135, in parse_et_yaml
    native_yaml = parse_native_yaml(
                  ^^^^^^^^^^^^^^^^^^
  File "$PYTHON3_11_LINUXx86_64_env/lib/python3.11/site-packages/torchgen/gen.py", line 241, in parse_native_yaml
    _GLOBAL_PARSE_NATIVE_YAML_CACHE[path] = parse_native_yaml_struct(
                                            ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "$PYTHON3_11_LINUXx86_64_env/lib/python3.11/site-packages/torchgen/gen.py", line 180, in parse_native_yaml_struct
    add_generated_native_functions(rs, bs)
  File "$PYTHON3_11_LINUXx86_64_env/lib/python3.11/site-packages/torchgen/native_function_generation.py", line 466, in add_generated_native_functions
    raise AssertionError(
AssertionError: Found an operator that we could not generate an out= variant for: _cslt_sparse_mm_search(Tensor compressed_A, Tensor dense_B, Tensor? bias=None, Tensor? alpha=None, ScalarType? out_dtype=None, bool transpose_result=False) -> int.
This type of operators don't have tensor-like return, making it difficult to generate a proper out= variant. If
out= variant is not needed, please add the function name into FUNCTIONAL_OPS_THAT_CANNOT_GET_AN_OUT_VARIANT list.
gmake[2]: *** [kernels/portable/CMakeFiles/portable_ops_lib.dir/build.make:144: kernels/portable/RegisterCodegenUnboxedKernelsEverything.cpp] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:322: kernels/portable/CMakeFiles/portable_ops_lib.dir/all] Error 2
gmake: *** [Makefile:136: all] Error 2

In both cases, it's the same file that throwing me an error, does anybody know why would this happen?

The build folder for Linux x 86_64 is located at $EXECUTORCH/build_x86_64
The build folder for android is located at $EXECUTORCH/build_android

[haowhsu-quic]

Hi, could you follow the instruction in https://github.com/pytorch/executorch/blob/main/backends/qualcomm/setup.md or try https://github.com/pytorch/executorch/blob/main/backends/qualcomm/scripts/build.sh as an alternative.

We will update the outdated documents, thanks.

[mergennachin]

@podraco

Were you able to follow documentations and instructions in the latest main branch as @haowhsu-quic recommended, by any chance? is it happening only if you try to do cross compilation to QNN? Does it happen if you just run on your locally without any QNN delegation?

cc @larryliu0820 in case you are interested in the out_variant error for _cslt_sparse_mm_search

[larryliu0820]

@podraco for the _cslt_sparse_mm_search I suspect you are not installing the correct version of torch. Can you run bash ./install_requirements.sh and make sure torch version is correct?

[podraco]

Thanks for your patience, i've been working in other tasks and time flew away, so far i'm using the backends/qualcomm/setup.md file.

There is a small error in the file, in step three
https://github.com/pytorch/executorch/blob/main/backends/qualcomm/setup.md#step-3-compile-a-model

The document should specify that the examples dir is located at $EXECUTORCH_ROOT/examples, this, due to the confusion to the examples dir located in $EXECUTORCH_ROOT/build_android/examples

i'm still doing some work and recompiling everything using the instructions @haowhsu-quic sent me.

i'll keep you updated if i find any issue, thanks!!!

[podraco]

Now it gets odd.

The command
python -m examples.qualcomm.scripts.export_example --model_name mv2

It is expected to be run at $EXECUTORCH_ROOT/.

But the file:
"$EXECUTORCH_ROOT/examples/qualcomm/scripts/export_example.py"

The file expects that the backends submodule is available from the executorch package.

Which is not yet installed anywhere. Only dirty available from $PYTHONPATH.

So, we should run this from $EXECUTORCH_ROOT/..
and add the executorch package name at the beginning of the call prior.

python -m executorch.examples.qualcomm.scripts.export_example --model_name mv2

after the small change, the command can run, but I face this "new" error:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "$EXECUTORCH_ROOT/examples/qualcomm/scripts/export_example.py", line 4, in <module>
    from executorch.backends.qualcomm.partition.qnn_partitioner import QnnPartitioner
  File "$EXECUTORCH_ROOT/backends/qualcomm/partition/qnn_partitioner.py", line 11, in <module>
    from executorch.backends.qualcomm.builders import node_visitor
  File "$EXECUTORCH_ROOT/backends/qualcomm/builders/__init__.py", line 7, in <module>
    from . import (
  File "$EXECUTORCH_ROOT/backends/qualcomm/builders/node_visitor.py", line 14, in <module>
    from executorch.exir.dialects._ops import ops as exir_ops
  File "$EXECUTORCH_ROOT/exir/__init__.py", line 9, in <module>
    from executorch.exir.capture import (
  File "$EXECUTORCH_ROOT/exir/capture/__init__.py", line 9, in <module>
    from executorch.exir.capture._capture import (
  File "$EXECUTORCH_ROOT/exir/capture/_capture.py", line 14, in <module>
    from executorch.exir.capture._config import CaptureConfig
  File "$EXECUTORCH_ROOT/exir/capture/_config.py", line 12, in <module>
    from executorch.exir.passes import MemoryPlanningPass, ToOutVarPass
  File "$EXECUTORCH_ROOT/exir/passes/__init__.py", line 19, in <module>
    from executorch.exir import control_flow, memory, memory_planning
  File "$EXECUTORCH_ROOT/exir/memory_planning.py", line 20, in <module>
    from executorch.exir.delegate import executorch_call_delegate
  File "$EXECUTORCH_ROOT/exir/delegate.py", line 119, in <module>
    @executorch_call_delegate.py_functionalize_impl
     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'HigherOrderOperator' object has no attribute 'py_functionalize_impl'

which may be due to a different version from torch, as larryliu0820 suggested. i'll look it up a little more.

[podraco]

so, i updated torch to the latest dev available and it seems it has higherorderoperator now, i believe the files never check the torch version, so we can't tell for sure the minimum version required to run them.

[mergennachin]

i believe the files never check the torch version

That's a good feedback. We will make enhancements to check for the version

[mergennachin]

Hey @podraco , regarding the higherorderoperator issue, a similar issue was reported and got fixed.

#1663

[mergennachin]

torch version is pinned now. Closing this issue.