[PYTHON] Find libraries in wheel-tagged build directories#19987
Conversation
There was a problem hiding this comment.
Code Review
This pull request updates the library search paths in python/tvm/libinfo.py to support the <worktree>/build/<wheel-tag>/lib/ layout by using glob to find matching directories. The reviewer suggested guarding the glob operation with a check to ensure the build directory exists, which prevents unnecessary filesystem scanning in non-development environments, and caching the result of _dev_top_directory() to avoid redundant calls.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| build = _dev_top_directory() / "build" | ||
| paths: list[Path] = [] | ||
| if os.environ.get("TVM_LIBRARY_PATH"): | ||
| paths.append(Path(os.environ["TVM_LIBRARY_PATH"])) | ||
| paths += [ | ||
| pkg / "lib", # wheel layout | ||
| _dev_top_directory() / "build" / "lib", # dev: <worktree>/build/lib | ||
| build / "lib", # dev: <worktree>/build/lib | ||
| *sorted(build.glob("*/lib")), # dev: <worktree>/build/<wheel-tag>/lib | ||
| _dev_top_directory() / "lib", # dev: <worktree>/lib | ||
| ] |
There was a problem hiding this comment.
Unconditionally calling build.glob("*/lib") can be inefficient and potentially problematic in non-development environments (e.g., when TVM is installed as a wheel and the build directory does not exist or refers to an unrelated system path). Since glob is evaluated immediately when package_lib_paths() is called (which happens during import tvm), we should guard this with build.is_dir() to avoid unnecessary filesystem scanning. Additionally, we can cache the result of _dev_top_directory() to avoid calling it twice.
| build = _dev_top_directory() / "build" | |
| paths: list[Path] = [] | |
| if os.environ.get("TVM_LIBRARY_PATH"): | |
| paths.append(Path(os.environ["TVM_LIBRARY_PATH"])) | |
| paths += [ | |
| pkg / "lib", # wheel layout | |
| _dev_top_directory() / "build" / "lib", # dev: <worktree>/build/lib | |
| build / "lib", # dev: <worktree>/build/lib | |
| *sorted(build.glob("*/lib")), # dev: <worktree>/build/<wheel-tag>/lib | |
| _dev_top_directory() / "lib", # dev: <worktree>/lib | |
| ] | |
| dev_top = _dev_top_directory() | |
| build = dev_top / "build" | |
| paths: list[Path] = [] | |
| if os.environ.get("TVM_LIBRARY_PATH"): | |
| paths.append(Path(os.environ["TVM_LIBRARY_PATH"])) | |
| paths += [ | |
| pkg / "lib", # wheel layout | |
| build / "lib", # dev: <worktree>/build/lib | |
| *(sorted(build.glob("*/lib")) if build.is_dir() else []), # dev: <worktree>/build/<wheel-tag>/lib | |
| dev_top / "lib", # dev: <worktree>/lib | |
| ] |
|
i see, i think in this case, it is perhaps better to simply remove wheel_tag from build dir. While wheel tag helps to distinguish multiple build, i dpn't think it is needed if we isolate CIBW for each wheel matrix, cc @tlopex |
|
@tqchen agreed. would send a pr to remove the wheel tag |
This PR removes `{wheel_tag}` from the scikit-build build directory,
solving the issue #19987 mentioned
Each cibuildwheel matrix job runs in an isolated environment, so
separate tagged build directories are unnecessary. Using `build`
restores the existing `build/lib` development layout expected by
`tvm.libinfo` and fixes editable installs without adding runtime
directory globbing.
|
Closing as #19990 got merged. |
#19656 changed the scikit-build build directory from
buildtobuild/{wheel_tag}.That change moved editable-build libraries from
build/libto paths such asbuild/py3-none-macosx_15_0_arm64/lib. However,tvm.libinfo.package_lib_paths()continued to search onlybuild/lib. As a result,import tvmcould not findlibtvm_runtimeafter an editablebuild:
This PR adds
build/*/libto the library search paths. It preserves the existing priority ofTVM_LIBRARY_PATH, wheel-installed libraries, andbuild/lib.Verified with:
uv run python -c "import tvm"