The Linux wheel's _portable_lib extension ships with absolute paths from the machine that built it in its runtime search path, including a path that exists on no user's machine, plus two Windows directories.
What a user gets
From a pip install executorch on Linux x86_64:
$ readelf -d .../executorch/extension/pybindings/_portable_lib.cpython-312-x86_64-linux-gnu.so
RUNPATH [$ORIGIN/../../../torch/lib
/lib/intel64
/lib/intel64_win
/lib/win-x64
/__w/_temp/conda_environment_21402829909/lib/python3.12/site-packages/torch/lib]
Four of those five entries are not intended:
/__w/_temp/conda_environment_<number>/... is an absolute path to the conda
environment on the build machine. It cannot exist for a user, and the number
changes from build to build, so it also makes the artifact non-reproducible.
/lib/intel64_win and /lib/win-x64 are Windows layouts, in a Linux wheel.
/lib/intel64 is a bare absolute path that resolves to whatever the user
happens to have there.
Only the first entry, $ORIGIN/../../../torch/lib, is deliberate and relocatable.
Where they come from
Torch's exported CMake package creates an MKL::MKL imported target, and linking
torch brings it in even though ExecuTorch never asks for MKL. The linker records
the directories it found MKL in. Packaging then copies the built extension out of
the build tree rather than running an install step, so whatever the linker
recorded ships as-is.
The build log makes the discovery visible:
-- MKL_ROOT /opt/conda/envs/py_3.10
-- MKL_ARCH: None, set to ` intel64` by default
-- MKL_INTERFACE_FULL: None, set to ` intel_ilp64` by default
Why it matters beyond untidiness
A build that resolves a dependency through one of those absolute entries is
resolving it through the layout of the build environment, not through anything
the wheel controls. That works on a machine whose filesystem happens to match and
fails elsewhere, and the failure appears at import time:
ImportError: libmkl_intel_ilp64.so.2: cannot open shared object file
So the paths are simultaneously useless to users and load-bearing for the build
environment, which is the worst combination: they cannot be removed without
noticing what was relying on them, and they cannot be relied on either.
Suggested direction
Two parts, and they are separable:
- Do not ship a runtime search path a user cannot use. An entry that is absolute
or names a foreign platform should not survive packaging. This is checkable:
read RUNPATH/RPATH on every shipped object and reject any absolute entry.
- Resolve MKL through something the wheel or its declared dependencies own,
rather than through a directory the build machine happened to have, so the
extension does not depend on the build environment's layout at load time.
Reproduction
pip install executorch
python - <<'EOF'
import executorch, pathlib, subprocess
p = next(pathlib.Path(executorch.__path__[0]).glob("extension/pybindings/_portable_lib*.so"))
print(subprocess.run(["readelf", "-d", str(p)], capture_output=True, text=True).stdout)
EOF
Any entry in the printed RUNPATH that starts with / is a path from the build
machine.
The Linux wheel's
_portable_libextension ships with absolute paths from the machine that built it in its runtime search path, including a path that exists on no user's machine, plus two Windows directories.What a user gets
From a
pip install executorchon Linux x86_64:Four of those five entries are not intended:
/__w/_temp/conda_environment_<number>/...is an absolute path to the condaenvironment on the build machine. It cannot exist for a user, and the number
changes from build to build, so it also makes the artifact non-reproducible.
/lib/intel64_winand/lib/win-x64are Windows layouts, in a Linux wheel./lib/intel64is a bare absolute path that resolves to whatever the userhappens to have there.
Only the first entry,
$ORIGIN/../../../torch/lib, is deliberate and relocatable.Where they come from
Torch's exported CMake package creates an
MKL::MKLimported target, and linkingtorch brings it in even though ExecuTorch never asks for MKL. The linker records
the directories it found MKL in. Packaging then copies the built extension out of
the build tree rather than running an install step, so whatever the linker
recorded ships as-is.
The build log makes the discovery visible:
Why it matters beyond untidiness
A build that resolves a dependency through one of those absolute entries is
resolving it through the layout of the build environment, not through anything
the wheel controls. That works on a machine whose filesystem happens to match and
fails elsewhere, and the failure appears at import time:
So the paths are simultaneously useless to users and load-bearing for the build
environment, which is the worst combination: they cannot be removed without
noticing what was relying on them, and they cannot be relied on either.
Suggested direction
Two parts, and they are separable:
or names a foreign platform should not survive packaging. This is checkable:
read
RUNPATH/RPATHon every shipped object and reject any absolute entry.rather than through a directory the build machine happened to have, so the
extension does not depend on the build environment's layout at load time.
Reproduction
Any entry in the printed
RUNPATHthat starts with/is a path from the buildmachine.