Skip to content

[BUILD] Modularize device runtime into per-backend DSOs#19594

Merged
spectrometerHBH merged 1 commit into
mainfrom
runtime-modularize
May 22, 2026
Merged

[BUILD] Modularize device runtime into per-backend DSOs#19594
spectrometerHBH merged 1 commit into
mainfrom
runtime-modularize

Conversation

@tqchen
Copy link
Copy Markdown
Member

@tqchen tqchen commented May 22, 2026

Summary

Modularize libtvm_runtime into per-backend shared libraries (libtvm_runtime_cuda, libtvm_runtime_vulkan, etc.) and libtvm_runtime_extra for contrib/disco modules.

  • Each backend can be built independently (USE_CUDA=ON produces only libtvm_runtime.so + libtvm_runtime_cuda.so)
  • During Python import, available backend DSOs are discovered and loaded automatically, with missing backends silently skipped
  • libtvm_runtime_extra is always produced if it has sources (disco, contrib modules)
  • No user-visible config flag changes — existing USE_CUDA, USE_VULKAN, etc. continue to work
  • Each contrib module builds as its own OBJECT lib, linked via target_link_libraries (modern CMake pattern)
  • CI artifact stashing updated to include new DSOs

Test plan

  • Build with USE_CUDA=ON — verify libtvm_runtime.so, libtvm_runtime_cuda.so, libtvm_runtime_extra.so produced
  • python -c "import tvm; print(tvm.cuda(0).exist)" prints True
  • Runtime GPU tests pass
  • Windows build passes (MSVC dllexport/dllimport)
  • cuBLAS/cuDNN JSON runtime functions registered via libtvm_runtime_extra

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request refactors the TVM build system to transition from a monolithic runtime to a modular structure using multiple shared libraries, including per-backend DSOs and a dedicated library for optional components. Key changes include updates to CMake modules for device runtimes, refined Python library loading logic, and adjusted CI artifact handling. Review feedback identifies a critical issue regarding duplicate symbol definitions in the CLML module due to redundant source inclusion and recommends more specific exception handling in the CI S3 script to prevent masking infrastructure failures.

Comment thread cmake/modules/contrib/CLML.cmake Outdated
Comment thread ci/scripts/jenkins/s3.py
@tqchen tqchen force-pushed the runtime-modularize branch 2 times, most recently from 3079d95 to f659c83 Compare May 22, 2026 04:27
Modularize libtvm_runtime into per-backend shared libraries
(libtvm_runtime_cuda, libtvm_runtime_vulkan, etc.) and
libtvm_runtime_extra for contrib/disco modules.

Each backend can be built independently. During Python import,
available backend DSOs are discovered and loaded automatically,
with missing backends silently skipped.
@tqchen tqchen force-pushed the runtime-modularize branch from f659c83 to e145b16 Compare May 22, 2026 15:03
@spectrometerHBH spectrometerHBH merged commit 4052880 into main May 22, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants