[pybind] Add back extension suffix for portable lib shared object in pip wheel

[larryliu0820]

Summary: Initially our portable lib prebuilt library is named as:

_portable_lib.cpython-310-x86_64-linux-gnu.so

Where it includes an EXT_SUFFIX cpython-310-x86_64-linux-gnu consists of architecture and build OS information.

This is enforced by setuptools following PEP 3149 and there's no good way to change this behavior.

#5961 was an attempt to rename the .so filename at packaging stage to _portable_lib.so, so that it is easier to be found by find_package() macro in CMake.

However #5961 is breaking the other prebuilt libraries such as libcustom_ops_aot_lib.so which depends on the original _portable_lib.cpython-310-x86_64-linux-gnu.so name in its RPATH.

This PR is a fix that reverts #5961 and restore the full name during packaging, but try to match the EXT_SUFFIX in CMake to be able to find the .so file.

Test Plan:

EXECUTORCH_BUILD_PYBIND=ON python setup.py bdist_wheel # build wheel including _portable_lib.*.so
pip install dist/executorch-0.5.0a0+7510f8c-cp310-cp310-linux_x86_64.whl
python -c "from executorch.extension.llm.custom_ops import sdpa_with_kv_cache"

# Run torchao script to make sure find_package() works
push ao/torchao/experimental
bash build_torchao_ops.sh executorch

Does not throw _portable_lib.cpython-310-x86_64-linux-gnu.so not found error.

Reviewers:

Subscribers:

Tasks:

Tags:

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/6363

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures

As of commit 8885cdb with merge base 7510f8c ():

NEW FAILURES - The following jobs have failed:

• pull / test-eval_llama-mmlu-linux / linux-job (gh)
  RuntimeError: Command docker exec -t d2c77fb20900c7911c05663f08c705d11f1df06039a9cfdac4f4f7c1e3d69def /exec failed with exit code 127
• pull / test-eval_llama-wikitext-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 686e6dbaa68f3a9e28edc6e50d56b01609cb79399f3196de4a29b670ab53e27d /exec failed with exit code 127
• pull / test-llama_runner_eager-linux / linux-job (gh)
  RuntimeError: Command docker exec -t e8e139f2cc04c68c65e04c97e1129a0fe46a23b5e8ecc98264207ade012df953 /exec failed with exit code 127

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[facebook-github-bot]

@larryliu0820 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[facebook-github-bot]

@larryliu0820 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

[facebook-github-bot]

@larryliu0820 merged this pull request in b3932c0.