Skip to content

[PYTHON] Find libraries in wheel-tagged build directories#19987

Closed
mshr-h wants to merge 1 commit into
apache:mainfrom
mshr-h:fix/tagged-build-library-path
Closed

[PYTHON] Find libraries in wheel-tagged build directories#19987
mshr-h wants to merge 1 commit into
apache:mainfrom
mshr-h:fix/tagged-build-library-path

Conversation

@mshr-h

@mshr-h mshr-h commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

#19656 changed the scikit-build build directory from build tobuild/{wheel_tag}.

That change moved editable-build libraries from build/lib to paths such as build/py3-none-macosx_15_0_arm64/lib. However, tvm.libinfo.package_lib_paths() continued to search only build/lib. As a result, import tvm could not find libtvm_runtime after an editable
build:

RuntimeError: Cannot find library libtvm_runtime.dylib, libtvm_runtime.so

This PR adds build/*/lib to the library search paths. It preserves the existing priority of TVM_LIBRARY_PATH, wheel-installed libraries, and build/lib.

Verified with:

uv run python -c "import tvm"

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the library search paths in python/tvm/libinfo.py to support the <worktree>/build/<wheel-tag>/lib/ layout by using glob to find matching directories. The reviewer suggested guarding the glob operation with a check to ensure the build directory exists, which prevents unnecessary filesystem scanning in non-development environments, and caching the result of _dev_top_directory() to avoid redundant calls.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread python/tvm/libinfo.py
Comment on lines +49 to 58
build = _dev_top_directory() / "build"
paths: list[Path] = []
if os.environ.get("TVM_LIBRARY_PATH"):
paths.append(Path(os.environ["TVM_LIBRARY_PATH"]))
paths += [
pkg / "lib", # wheel layout
_dev_top_directory() / "build" / "lib", # dev: <worktree>/build/lib
build / "lib", # dev: <worktree>/build/lib
*sorted(build.glob("*/lib")), # dev: <worktree>/build/<wheel-tag>/lib
_dev_top_directory() / "lib", # dev: <worktree>/lib
]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Unconditionally calling build.glob("*/lib") can be inefficient and potentially problematic in non-development environments (e.g., when TVM is installed as a wheel and the build directory does not exist or refers to an unrelated system path). Since glob is evaluated immediately when package_lib_paths() is called (which happens during import tvm), we should guard this with build.is_dir() to avoid unnecessary filesystem scanning. Additionally, we can cache the result of _dev_top_directory() to avoid calling it twice.

Suggested change
build = _dev_top_directory() / "build"
paths: list[Path] = []
if os.environ.get("TVM_LIBRARY_PATH"):
paths.append(Path(os.environ["TVM_LIBRARY_PATH"]))
paths += [
pkg / "lib", # wheel layout
_dev_top_directory() / "build" / "lib", # dev: <worktree>/build/lib
build / "lib", # dev: <worktree>/build/lib
*sorted(build.glob("*/lib")), # dev: <worktree>/build/<wheel-tag>/lib
_dev_top_directory() / "lib", # dev: <worktree>/lib
]
dev_top = _dev_top_directory()
build = dev_top / "build"
paths: list[Path] = []
if os.environ.get("TVM_LIBRARY_PATH"):
paths.append(Path(os.environ["TVM_LIBRARY_PATH"]))
paths += [
pkg / "lib", # wheel layout
build / "lib", # dev: <worktree>/build/lib
*(sorted(build.glob("*/lib")) if build.is_dir() else []), # dev: <worktree>/build/<wheel-tag>/lib
dev_top / "lib", # dev: <worktree>/lib
]

@tqchen

tqchen commented Jul 12, 2026

Copy link
Copy Markdown
Member

i see, i think in this case, it is perhaps better to simply remove wheel_tag from build dir. While wheel tag helps to distinguish multiple build, i dpn't think it is needed if we isolate CIBW for each wheel matrix, cc @tlopex

@tlopex

tlopex commented Jul 12, 2026

Copy link
Copy Markdown
Member

@tqchen agreed. would send a pr to remove the wheel tag

mshr-h pushed a commit that referenced this pull request Jul 13, 2026
This PR removes `{wheel_tag}` from the scikit-build build directory,
solving the issue #19987 mentioned

Each cibuildwheel matrix job runs in an isolated environment, so
separate tagged build directories are unnecessary. Using `build`
restores the existing `build/lib` development layout expected by
`tvm.libinfo` and fixes editable installs without adding runtime
directory globbing.
@mshr-h

mshr-h commented Jul 13, 2026

Copy link
Copy Markdown
Contributor Author

Closing as #19990 got merged.

@mshr-h mshr-h closed this Jul 13, 2026
@mshr-h mshr-h deleted the fix/tagged-build-library-path branch July 13, 2026 02:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants